我需要建立敏感的数据传输/存储/加密系统的建议 [英] I need advice developing a sensitive data transfer/storage/encryption system
问题描述
简介
我目前正在从事VisualFox Pro数据库日常提取数据（药房记录）的项目，并将其中的一些内容上传到WordPress网站，药房的客户可以安全地查看。我想要一些关于我的软件的一般方法的建议 - 我可以编写它，但需要知道我是否正确的方法。我正在编写PC软件（在C＃/。NET 4.5）和PHP WordPress插件。
问题1：加密
我计划使用的加密数据服务器端的当前进程是基于这篇文章。总而言之，它主张使用自己的公钥不对称地对每个单独的用户的数据进行加密，存储在服务器上。然后，使用用户密码对称地对该数据进行解密的私钥进行加密并存储。这样，即使数据库被盗，用户的密码哈希也需要被破坏，即使这个过程需要重复每个用户的数据。
作者本人指出的弱点，我的问题的要点是，用户登录时，解密的密钥存储在会话存储中。文章建议处理它的方法是限制用户登录的时间。我认为一个更好的解决方案是将该密钥存储在短命的安全cookie中（当然整个过程都是通过HTTPS发生的） 。这样一来，如果攻击者控制了用户的电脑，并且可以读取他们的cookies，那么他们可能只需键入密码即可登录，无需窃取数据库，即使攻击者可以访问服务器，也无法解密HTTPS流量（或者可以吗？我不确定。）
我应该使用安全Cookie或会话存储来临时存储解密密钥吗？ strong>
问题2：存储
我还想要解决的第二件事是如何存储数据 - 这更是一个效率问题。由于每个用户都有自己的加密密钥，因此每个用户的记录必须分开存储。我不知道是否应该为每个用户存储一个块数据，其中包含加密的JSON与表示记录的对象数组，或者是否应该将记录存储在具有实际数据结构的表中，并加密每个数据字段单独使用密钥。
我倾向于将数据存储为一个块 - 在我看来，一次解密一个大块数据似乎更有效率，比可能数千个不同的领域。另外，即使我将数据存储在正确的结构中，我仍然无法使用MySQL的WHERE，ORDERBY等，因为这些数据都是BLOB。
我应该将数据存储为每个用户的大块，还是分隔成不同的字段？
问题3：传输
我从DBF文件中提取数据，本质上是一个差异，从而比较了最后一天的数据提取的数据，只上传用户的块已经改变（我不仅可以上传记录，因为我可能会最终将用户的数据存储在块中）。我还包括已删除的用户的删除指令。这是因为数据库中有数十万条记录，总计超过200mb，大小每天都在增加。
我目前的计划是将所有这些数据写入JSON文件，gzip并将其上传到服务器。我的问题是，如何在确保数据的安全性的同时做到这一点？当然，上传将通过HTTPS发生，我有一个API密码，只允许授权的上传，但我的主要关注是如果服务器受到威胁，如何保护数据。我不希望攻击者正在处理服务器时抓住JSON文件。一个想法是让服务器向我发送一个用户的公钥列表，并在上传前在我的软件中执行加密。在我看来，这是保护数据的唯一方法。我可以使用API密钥或特殊密码对整个JSON文件进行加密，但是如果攻击者可以在服务器上正在处理解密的文件时才能访问该文件。这是一个很好的解决方案吗？
我应该如何单独加密客户端的数据，还是有办法安全地将其传输到服务器并对其进行加密？
提前感谢任何答案，我很乐意听到有人处理过这样的问题。
注意：交叉发布到程序员 ，请参阅评论。
问题1
加密
正因为如此，我正在使用类似的系统加密个人详细信息（电子邮件，IP）在Wordpress注释中，以便如果服务器遭到入侵，数据库中的敏感数据仍然被加密。在会议中存储一个不对称的解密密钥已经出来了，因为这可能会让服务器上的密钥在攻击者的同时遭到破坏。
$ b $如您所说，加密记录（根据我的设计使用一个密钥，或根据您的设计使用一个密钥）意味着搜索记录将成为您必须从数据库服务器移开的过程，这反过来意味着这将显着变慢。
您可以通过在速度和安全性之间进行权衡来减轻这种情况：某些字段可以模糊，然后存储未加密。例如，如果要搜索患者所在的位置，请从其地址获取（纬度，长度），然后对其进行随机移位（在两个方向上最多可达3英里），然后存储所得到的坐标在纯文本。关于位置的近似计数查询可以在不解密的情况下完成。
减轻对客户端计算机的攻击
以上是关于如何缓解针对服务器的攻击，这是您最大的风险，因为您将所有的记录存储在那里。正如你正确地指出的那样，对客户机的攻击也是一个问题，如果他们是公众成员，那么他们的安全过程可以被认为是不存在的。
在此基础上，您可以使用密码来加强单个密码（其全部内容），客户端需要从其中选择三个随机字母（即特别未全部给出）。这样可以通过两种方式来优化对抗键盘记录器：首先使用下拉式菜单，这很难被窃听，即使用户使用键盘快捷键，也没有提供完整的短语。在每次成功登录时，记录随机字母（例如1,4和5）的索引，并且不会再长时间询问。显然，太多错误的答案导致帐户被锁定，需要通过电话或蜗牛邮件重置代码进行重新授权。
您可以使用的其他身份验证方法：text用户每次输入正确的密码时都会有额外的密码，或者（可能非常昂贵）使用网络银行的身份验证设备。
存储很少/不存在识别信息
另一个安全提示是尽可能少的存储个人信息。如果您无法通过电子邮件立即重置密码，则可能不需要姓名，地址，电话号码和电子邮件 - 所有个人识别数据。该个人信息可以单独存储在另一个服务器上的断开数据库上，使用通用主键将它们链接在一起。 （事实上，如果用户希望重置密码，您可以简单地在其匿名用户记录上存储一个标志，药剂师可以在他们的防火墙机器上下次访问管理面板时手动执行重置过程）。
问题2
您应该加密一个blob中的表格数据还是将其留在每列中？我在应用程序中也看过这个。对我来说，我将它存储在一个blob中，因为我的用例是搜索密集型的，并且每行N个解密，而不是使决策变得容易。也就是说，你可能更喜欢单独加密列的整洁，有人可以认为，如果腐败蔓延，将它们分开可以让你有更好的机会让一些行能够生存。
如果您决定存储在一个blob中，我使用的格式与此类似（在不对称加密之前用换行分隔的行）：
1.2< - 这种格式的版本，所以我可以在以后添加东西
key1 = value1
key2 = value2
...
如果您有多个进程写入列，请确保在读写之间锁定行，否则（如上所述）您可能会丢失一些数据。
正如你所说，这可能同样是JSON，如果这种格式更适合你。
问题3
我对此问题的理解是：如果您无法自己解密用户记录，您如何复制到未加密的脱机副本？我在这里想知道您是否可以放松您的安全约束，并在服务器上存储公用公钥，并保留使用公共密钥加密的更改的单独记录。这将填充一个应定期清空的表（通过在远程安全机器上运行同步程序）;因此，与获取未加密的整个数据库相比，更改表对攻击者的值将很小。
当然，相应的私钥应该在药剂师的电脑，再次从网络上安全地防火。
这种设计的风险在于攻击者用他/她自己的一个替换了服务器公钥，以便他们以后可以收集有效地为他们加密的信息！但是，只要您在服务器上安装了跳线，就可以合理地防范：如果这被触发，Web应用程序的动态部分将不会写入任何新的更改（实际上不会起作用）直到系统被扫描并确定为安全。
Intro
I'm currently working on a project which involves the daily extraction of data (pharmacy records) from a VisualFox Pro database, and uploading some of it to a WordPress site, where clients of the pharmacy can securely view it. I would like some advice in terms of the general methodology of my software - I am able to code it, but need to know if I'm going the right way about it. I'm writing both the PC software (in C#/.NET 4.5) and the PHP WordPress plugin.
Question 1: Encryption
The current process for encrypting the data server-side I plan to use is based on this article. Summarised, it advocates encrypting each separate user's data asymmetrically with their own public key, stored on the server. The private key to decrypt this data is then itself encrypted symmetrically using the user's password, and stored. This way, even if the database is stolen, the user's password hash needs to be broken, and even then the process needs to be repeated for every user's data.
The only weakness, pointed out by the author himself, and the main point of my question, is the fact that while the user is logged in, the decrypted key is stored in session storage. The way the article suggests to deal with it is to just limit the time the user is logged in. I thought a better solution would be to store that key in a short-lived secure cookie (of course the whole process is happening over HTTPS). That way, if the attacker has control of the user's computer and can read their cookies, they can probably just keylog their password and log in, no need to steal the database, while even if the attacker gains access to the server, they cannot decrypt the HTTPS traffic (or can they? I'm not sure.)
Should I use secure cookies or session storage to temporarily store the decrypted key?
Question 2: Storage
The second thing I still want to work out is how to store the data - this is more of an efficiency problem. Since every user has their own key for encryption, it follows that records for every user must be stored separately. I don't know if I should store a "block" of data for every user, containing encrypted JSON with an array of objects representing records, or whether I should store records in a table with the actual data structure, and encrypt each data field separately with the key.
I am leaning towards storing the data as one block - it seems to me to be more efficient to decrypt one big block of data at a time, than perhaps several thousands separate fields. Also, even if I stored the data in its proper structure, I still wouldn't be able to use MySQL's WHERE, ORDERBY etc, since the data would all be BLOBs.
Should I store the data as a big block per user, or separated into the different fields?
Question 3: Transfer
I extract the data from the DBF file, and essentially make a "diff", whereby I compare the current extracted data from the last day's data, and only upload the blocks of the users that have changed (I can't only upload the records, as I probably will end up storing the users' data in blocks). I also include "delete" instructions for users which have been deleted. This is as there are hundreds of thousands records in the database, totalling over 200mb, and the size increases every day.
My current plan is to write all this data to a JSON file, gzip it and upload it to the server. My question is, how do I do that while ensuring the security of the data? Naturally, the upload will happen over HTTPS, and I have an API password in place to only allow authorised uploads, but my main concern is how to protect the data if the server is compromised. I don't want the attacker to just grab the JSON file from the server while it's being processed. One idea I had was to get the server to send me a list of public keys for the users, and perform the encryption in my software, before the upload. It seems to me like that's the only way of protecting that data. I could encrypt the whole JSON file, perhaps with an API key or a special password, but that's moot if the attacker can just access the decrypted file as it's being processed on the server. Is that a good solution?
Should I encrypt the data individually client-side, or is there a way to securely transfer it to the server and encrypt it there?
Thanks in advance for any answers, I'd love to hear from someone who's dealt with problems like this before.
Note: cross-posted to Programmers, see comments.
Question 1
Encryption
As it happens, I am working on a similar system to encrypt personal details (email, IP) in Wordpress comments, so that if the server is compromised, sensitive data in the database is still encrypted. Storing an assymetric decryption key in the session was out for me, since this could leave the key on the server for an attacker to grab at the same time as their compromising it.
So, cookies over an SSL cert is a better way to go - at least the attacker then has to wait for a user to log in before they can steal their key(s). In tandem with this, some sort of tripwire system would be a good idea, so that users cannot log onto the system (thus providing their keys to the waiting attacker) once it is compromised.
As you say, encrypting records (either with one key as per my design, or many keys as per yours) means that searching through records becomes a process you have to move away from your database server, which in turns means that it will be significantly slower.
You may be able to mitigate against this by making a trade-off between speed and security: some fields can be fuzzied and then stored unencrypted. For example, if you want to search where your patients are located, get their (lat, long) from their address, apply a random shift to it (say up to 3 miles on both axes in either direction) and then store the resulting coordinates in plain text. Approximate count queries relating to location can then be done without decryption.
Mitigating against attacks on a client computer
The above looks at how to mitigate against attacks against the server, which is your greatest risk, since you have all your records stored there. As you rightly point out though, attacks on client machines is also a concern, and if they are members of the public then their security processes can be assumed to be non-existent.
On that basis you could strengthen a single password (which is given in its entirety) with a passphrase from which the client needs to select three random letters (i.e. it is specifically not given in its entirety). This defends elegantly against keyloggers in two ways: firstly drop-down menus are used, which are harder to eavesdrop, and even if the user uses keyboard shortcuts, they have not supplied the full phrase. At each successful logon, the index of the random letters (e.g. 1, 4 and 5) is recorded and not asked again for a long period. Obviously, too many wrong answers causes the account to be locked out and require reauthorisation via a phone call or snail-mail reset code.
Other authentication methods you could use: text the user an additional passphrase every time they enter the correct password, or (probably prohibitively expensive) use an authentication device as per online banking.
Store little/no identifying information
Another tip for security is to store as little personal information as possible. If you can do without the ability to immediately reset passwords via email, then name, address, telephone numbers and email - all personally identifying data - are perhaps unnecessary. That personal information can be stored separately on a disconnected database on another server, using a common primary key to link them together. (In fact if the user wishes to reset their password, you could simply store a flag against their anonymous user record, and the pharmacist can run the reset process manually on their firewalled machine when they next visit an admin panel).
Question 2
Should you encrypt tabular data in one blob or leave it in each column? I've looked at this one as well in my application. For me, I stored it in one blob, since my use-case is search-intensive, and having N decrypts per row rather than one made the decision easy. That said, you may prefer the tidiness of encrypting columns individually, and one could argue that if corruption creeps in, separating them out gives you a better chance that some of the row will survive.
If you decide to store in a single blob, I am using a format similar to this (rows separated with newlines prior to being asymmetrically encrypted):
1.2 <-- version of this format, so I can add things in the future
key1=value1
key2=value2
...
If you have several processes writing to columns, make sure you lock rows between read and write, otherwise (as hinted above) you can lose some of your data.
As you say, this could equally be JSON, if that format is better for you.
Question 3
My understanding of this question is: how do you replicate to an unencrypted offline copy given that you cannot decrypt user records yourself? I wonder here whether you could relax your security constraints a bit, and store a common public key on the server, and keep a separate record of changes encrypted with the common key. This would populate a table which should periodically be emptied (by running a sync routine on a remote secure machine); thus, the value of the changes table to an attacker will be small compared to obtaining the whole database unencrypted.
The corresponding private key, of course, should be on the pharmacist's computer, again securely fire-walled from the internet.
The risk with this design is that an attacker replaces the server public key with one of his/her own, so that they can later collect information that has effectively been encrypted just for them! However, as long as you've installed a trip-wire on the server, this can be reasonably defended against: if this is triggered, the dynamic part of the web application won't write any new changes (in fact won't work at all) until the system is scanned and determined to be safe.
这篇关于我需要建立敏感的数据传输/存储/加密系统的建议的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
