48 replies to this topic

[+theblazingangel]

SPEhosting, on 20 November 2012 - 18:05, said:

well then maybe you can help me..... I need to create a p2p chat system with encryption.... BUT the encryption key obviously cant be in the source code or sent over the internet in plain text ..... any helps ?

You need to step away from the code, and properly think about how the overall design will work.

Are you producing a client for existing protocols?

I get that this is probably just a personal programming exercise, but are you creating client software that is based on one or more existing chat protocols and infrastructures, and just throws encryption on top? Or are you creating entirely your own thing? I'm going to assume the latter!

So, will there be a web service?

How will users discover each other? How will they know when each other is online? And what about authentication?

While it would be possible to completely avoid having a web service (and maybe that's what you actually meant by 'p2p'), it would be a pain to use:

• To connect to each other, users would have to communicate their IP address and port number to each other through some other means, and then enter this information into their chat clients.
  
• Some people have dynamic IP addresses, and additionally the port number may not be fixed, so recording this information in a 'friends list' would be useless and therefore knowing whether each other is online without separately speaking to each other would be impossible.
  
• If a user's IP address is dynamic and changes part way through a chat, their new IP address is going to have to be provided to the other user all over again, and a portion of the chat may have been lost in the interruption.
  
• NAT could complicate things even more.

A third-party dynamic DNS service could perhaps make some of this easier, but adds problems of its own, and there's a better option - a centralised web service.

A centralised web service will allow users to connect to one another in a simple and clean manner.

• Users will create a unique alias on the web service and then enter it into their client. The client software can automatically talk to the web service to provide/update the IP address and port number to associate with it.
  
• Periodic checks by the service, or "check ins" generated by the client, are done to keep track of the user's connection status.
  
• Tying a password to the alias prevents identity theft / impersonation.
  
• One user would still need to disclose a piece of information (their alias) to the other user in order to create the initial connection, but there's no getting around that. Thankfully this way is much better than above though, and with a connection established, the users can be recorded in each other's 'friends list' and they never have to supply it again. If you think your users would accept it, you could even offer an email search/lookup facility, with a friend request mechanism.

A couple of notes:

• In storing a friend list, the unique ID (aka UID, normally a number) should be recorded (hidden), not the alias, to allow users to change their aliases without breaking friend list entries.
  
• If one user decides to remove someone from their friend list, you may want to consider automatically removing them from the other person's too.

So how to go about encrypting chat then?

As already pointed out by others, asymmetric encryption is the best way of implementing this, and I hope and assume that you're already familiar with it. We need to think about some specifics though!

One thing that may influence how you implement encryption will be legislation (if we're pretending that you were developing a real product here). Your government may not actually allow you to produce an encrypted chat mechanism with no means what-so-ever of allowing them to snoop on it. Let's pretend that there would be no such restrictions though.

One simple way of implementing asymmetric encryption could be by using an encryption key belonging to the web service as a 'legitimate' middle man, and dynamically creating client certificates on the fly. A copy of the web service's public key could be embedded in the client application (preventing a third-party middle man attack in transmitting it). When connecting to the service, the client creates a new certificate, encrypts the public key with the web service public key, and sends it to the web service, which then sends back an encrypted confirmation. In transmitting a message to the other user, the client encrypts it and sends it to the web service in the same way. The web service decrypts it, re-encrypts it with the public key for the other user, and sends it on.

There are two huge problems here though:

• The huge load placed on the web service. This could very easily be solved by only using the above mechanism to transfer the user's public keys securely to each other, then they can send encrypted communications between themselves, but does not solve the next problem below.
  
• The web service is a huge weak spot. Administrators of the web service can snoop at any time they like. The government can demand to be provided access to be able to snoop. If someone should hack the web service, they can snoop.

This kind of problem exists in a lot of systems out there. Drop box for instance, there's nothing really stopping administrators accessing your data unless you pre-encrypt it, which is a pain. HTTPS is a complete joke, and so therefore is S/MIME which afaik is based on it. There are a couple of excellent solutions though that we could derive inspiration from:

• One is spideroak, a competitor to dropbox (no, before you ask, I don't work for them). Spideroak encrypts all of your data, and keep a copy of the encryption key, but they never keep a copy of the password for the encryption key. If you loose your password, you loose your data. As long as they are true to their word, and the software really operates as they say it does, never sending your password to them or anyone else, your data is completely secure.
  
• PGP based email encryption. This is asymmetric encryption. Each user generates a key pair. They send a copy of their public keys to each other, and verify them (to ensure no-one has intercepted them and performed a switch) through another form of communication. This is completely secure as long as correct verification is done, and they keep their private keys secure.

We could improve the security of our chat application by copying PGP. (I think there's actually an existing plugin for Pidgin that does this). The client allows the user to generate a key pair, and the public key could automatically be sent to anyone you connect to. You use another means such as the phone to verify them, and you're secure. Public keys could also be stored in friend lists, and signed by the user, in order to record the fact that they have verified and can trust that key, so you don't need to verify it every time, and so that if the developer of the client (you) tried to switch the key to snoop, they'd notice. Users would still need to trust that the client application isn't leaking unencrypted chat content or their private keys back to the developer (you) or government, but it wouldn't be too difficult for an expert to analyse the binary of your application and the web traffic it generates in order to determine if anything fishy is going on. (It would not be possible to analyse the web service described earlier is this way, an analyst would have to be granted special access to it, and you could easily hide things or change them at any time you liked).

The design outlined is not necessarily perfect however, because the user's key is stored locally on their computer. If they want to use a different computer, or loose their computer for any reason, that's a problem for most users. It would probably be better if user encryption keys (public and private) were stored on the web service. The public key could be stored as is, just like a public public-key server, and record signatures placed on keys, allowing groups of friends to more easily establish trust within their group with fewer external verification checks needed. The private keys would be encrypted with the user's password, and the password would never itself be stored by the web service, just like spideroak. When the user logs in to their chat client, a mechanism is gone through to that authenticates the user, crucially without their password being submitted to the web service, and hopefully without unnecessarily handing out a copy of the private key to anyone without the correct password. This sounds difficult if not impossible but spideroak apparently manages to do exactly this! Additionaly the transfer of the private key once the user is authenticated must be done securely, perhaps the copy of the already encrypted copy could be sent, and then decrypted with the password in the client...but then why wouldn't spideroak simply do that...I think I need to get some sleep at this point, and think this bit through some other time...

One potential problem with this improved mechanism though is that for security, users cannot log in to the website (pretending one existed for the product), if logging in to it was needed for some particular functionality, without compromising their security. Spideroak strongly advise against logging in to the website, instead doing everything within, or establishing an authenticated web session through, their application.

I'm probably going way beyond what you perhaps wanted with this, but it was interesting to think about

What about saved chats?

This can wait for another time, It's really late now and I should get some sleep...!

-----
edit: fixed a few minor typos and a broken link

[SPEhosting]

theblazingangel, on 21 November 2012 - 07:17, said:

You need to step away from the code, and properly think about how the overall design will work.

Are you producing a client for existing protocols?

I get that this is probably just a personal programming exercise, but are you creating client software that is based on one or more existing chat protocols and infrastructures, and just throws encryption on top? Or are you creating entirely your own thing? I'm going to assume the latter!

So, will there be a web service?

How will users discover each other? How will they know when each other is online? And what about authentication?

While it would be possible to completely avoid having a web service (and maybe that's what you actually meant by 'p2p'), it would be a pain to use:

• To connect to each other, users would have to communicate their IP address and port number to each other through some other means, and then enter this information into their chat clients.
  
• Some people have dynamic IP addresses, and additionally the port number may not be fixed, so recording this information in a 'friends list' would be useless and therefore knowing whether each other is online without separately speaking to each other would be impossible.
  
• If a user's IP address is dynamic and changes part way through a chat, their new IP address is going to have to be provided to the other user all over again, and a portion of the chat may have been lost in the interruption.
  
• NAT could complicate things even more.

A third-party dynamic DNS service could perhaps make some of this easier, but adds problems of its own, and there's a better option - a centralised web service.

A centralised web service will allow users to connect to one another in a simple and clean manner.

• Users will create a unique alias on the web service and then enter it into their client. The client software can automatically talk to the web service to provide/update the IP address and port number to associate with it.
  
• Periodic checks by the service, or "check ins" generated by the client, are done to keep track of the user's connection status.
  
• Tying a password to the alias prevents identity theft / impersonation.
  
• One user would still need to disclose a piece of information (their alias) to the other user in order to create the initial connection, but there's no getting around that. Thankfully this way is much better than above though, and with a connection established, the users can be recorded in each other's 'friends list' and they never have to supply it again. If you think your users would accept it, you could even offer an email search/lookup facility, with a friend request mechanism.

A couple of notes:

• In storing a friend list, the unique ID (aka UID, normally a number) should be recorded (hidden), not the alias, to allow users to change their aliases without breaking friend list entries.
  
• If one user decides to remove someone from their friend list, you may want to consider automatically removing them from the other person's too.

So how to go about encrypting chat then?

As already pointed out by others, asymmetric encryption is the best way of implementing this, and I hope and assume that you're already familiar with it. We need to think about some specifics though though!

One thing that may influence how you implement encryption will be legislation (if we're pretending that you were developing a real product here). Your government may not actually allow you to produce an encrypted chat mechanism with no means what-so-ever of allowing them to snoop on it. Let's pretend that there would be no such restrictions though.

One simple way of implementing asymmetric encryption could be by using an encryption key belonging to the web service as a 'legitimate' middle man, and dynamically creating client certificates on the fly. A copy of the web service's public key could be embedded in the client application (preventing a third-party middle man attack in transmitting it). When connecting to the service, the client creates a new certificate, encrypts the public key with the web service public key, and sends it to the web service, which then sends back an encrypted confirmation. In transmitting a message to the other user, the client encrypts it and sends it to the web service in the same way. The web service decrypts it, re-encrypts it with the public key for the other user, and sends it on.

There are two huge problems here though:

• The huge load placed on the web service. This could very easily be solved by only using the above mechanism to transfer the user's public keys securely to each other, then they can send encrypted communications between themselves, but does not solve the next problem below.
  
• The web service is a huge weak spot. Administrators of the web service can snoop at any time they like. The government can demand to be provided access to be able to snoop. If someone should hack the web service, they can snoop.

This kind of problem exists in a lot of systems out there. Drop box for instance, there's nothing really stopping administrators accessing your data unless you pre-encrypt it, which is a pain. HTTPS is a complete joke, and so therefore is S/MIME which afaik is based on it. There are a couple of excellent solutions though that we could derive inspiration from:

• One is spideroak, a competitor to dropbox (no, before you ask, I don't work for them). Spideroak encrypts all of your data, and keep a copy of the encryption key, but they never keep a copy of the password for the encryption key. If you loose your password, you loose your data. As long as they are true to their word, and the software really operates as they say it does, never sending your password to them or anyone else, your data is completely secure.
  
• PGP based email encryption. This is asymmetric encryption. Each user generates a key pair. They send a copy of their public keys to each other, and verify them (to ensure no-one has intercepted them and performed a switch) through another form of communication. This is completely secure as long as correct verification is done, and they keep their private keys secure.

We could improve the security of our chat application by copying PGP. (I think there's actually an existing plugin for Pidgin that does this). The client allows the user to generate a key pair, and the public key could automatically be sent to anyone you connect to. You use another means such as the phone to verify them, and your secure. Public keys could also be stored in friend lists, and signed by the user, in order to record the fact that they have verified and can trust that key, so you don;t need to verify it every time, and so that if the developer of the client (you) tried to switch the key to snoop, they'd notice. Users would still need to trust that the client application isn't leaking unencrypted chat content or their private keys back to the developer (you) or government, but it wouldn't be too difficult for an expert to analyse the binary of your application and the web traffic it generates in order to determine if anything fishy is going on. (It would not be possible to analyse the web service described earlier is this way, an analyst would have to be granted special access to it, and you could easily hide things or change them at any time you liked).

The design outlined is not necessarily perfect however, because the user's key is stored locally on their computer. If they want to use a different computer, or loose their computer for any reason, that's a problem for most users. It would probably be better if user encryption keys (public and private) were stored on the web service. The public key could be stored as is, just like a public public-key server, and record signatures placed on keys, allowing groups of friends to more easily establish trust within their group with fewer external verification checks needed. The private keys would be encrypted with the user's password, and the password would never itself be stored by the web service, just like spideroak. When the user logs in to their chat client, a mechanism is gone through to that authenticates the user, crucially without their password being submitted to the web service, and hopefully without unnecessarily handing out a copy of the private key to anyone without the correct password. This sounds difficult if not impossible but spideroak https://spideroak.co...do exactly this! Additionaly the transfer of the private key once the user is authenticated must be done securely, perhaps the copy of the already encrypted copy could be sent, and then decrypted with the password in the client...but then why wouldn't spideroak simply do that...I think I need to get some sleep at this point, and think this bet through some other time...

One potential problem with this improved mechanism though is that for security, users cannot log in to the website (pretending one existed for the product), if logging in to it was needed for some particular functionality, without compromising their security. Spideroak strongly advise against logging in to the website, instead doing everything within, or establishing an authenticated web session through, their application.

I'm probably going way beyond what you perhaps wanted with this, but it was interesting to think about

What about saved chats?

This can wait for another time, It's really late now and I should get some sleep...!

well I will answer the first two sub titles the 3rd one will require more reading but I have to get my washing out before i go to class xD ......

firstly thank you for taking the time to show interest it shows you are taking me seriously...

secondly This is my aim and how I am trying to get there...

this P2P connection will have a host so far im starting off easy, the host will be decided by the people using the chat (e.g. run host.class)

the host will be listening on the desired port (default probs 4444 or 5555 something like that) the client then connects to host ... yes target will need to be specified by client (I am trying to do this with as little database work as possible but I am not closed to it )

this program is not designed for use by a large populous its a program that will be off the radar for now and used for specific private communications....

I was thinking about using private and public key methods but im not sure how they work yet....

There will be another chat I will be making in the future which will use usernames and such both with encryption and without (one is a project I have to do other is just because I want to ) ....

like I said rest will have to wait!!! also no saved chats ever ! all will be burnt ...

[n_K]

Yes theblazingangel, that's what I was talking about.
SPEhosting if you fail to plan, you plan to fail. Always plan and you can see how or if things will work

[Salutary7]

SPEhosting, on 20 November 2012 - 15:19, said:

even when I encrypt with DES both times i still get the same result ...the thing is im trying to set up a safe way to send information where neither party has the others keys so encryptions needs to be removed in the same order they were added .... so if i encrypt the string "moo" ... with a passkey such as "hello" then encrypt the new string with the passkey "goodbye" I then need to remove the encryption "hello" after which i then remove the passkey goodbye .... get it?

What you're talking about is mathematically possible, however I don't think it can be secure. The only way (that I know of) to do this is is with very simple algorithms (hence the lack of security). Take, for example the XOR bitwise operation. With a simple XOR cipher, you can encrypt a text with a key K1, and then again with a key K2. To decrypt, you can use K1 and K2 in any order.

I would, however, suggest following the lead of existing open source software that fits with your goals. Given your current knowledge, it's basically impossible to invent a novel cryptographic approach.

If I have seen further it is by standing on the shoulders of giants. --Isaac Newton

[SPEhosting]

Salutary7, on 21 November 2012 - 19:33, said:

What you're talking about is mathematically possible, however I don't think it can be secure. The only way (that I know of) to do this is is with very simple algorithms (hence the lack of security). Take, for example the XOR bitwise operation. With a simple XOR cipher, you can encrypt a text with a key K1, and then again with a key K2. To decrypt, you can use K1 and K2 in any order.

I would, however, suggest following the lead of existing open source software that fits with your goals. Given your current knowledge, it's basically impossible to invent a novel cryptographic approach.

If I have seen further it is by standing on the shoulders of giants. --Isaac Newton

ye I know it is possible I went to a seminar about this kind of stuff ... though we talked about methods and ideas they never really shared how to do it xD which is annoying so I know its possible these people are contractors for the DoD ....I am going to try public and private keys to see where that gets me though I dont want to use it without knowing exactly how the mathmatical algorithms work it seems like an odd concept ... once I read the logic behind it i will be fine ....

"with great power comes great responsibility" -- ben parker

[+Asik]

theblazingangel, on 21 November 2012 - 07:17, said:

[snip]

best post in this entire forum since a long while.

[SPEhosting]

Dr_Asik, on 21 November 2012 - 20:48, said:

best post in this entire forum since a long while.

I know right someone who shows interest and posts a length post !

also question for you!

I cannot understand why this is refusing to work

KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
kpg.initialize(1024); //it complains about this line, "miss placed constructs, delete token 1024? 
KeyPair kp = kpg.genKeyPair();
Key publicKey = kp.getPublic();
Key privateKey = kp.getPrivate();

[+Asik]

Is that code inside a method?

[SPEhosting]

package enchat;
import java.security.*;
import java.security.spec.*;
import java.io.*;
import java.math.*;
public class RSAe {

KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
kpg.initialize(1024);
KeyPair kp = kpg.genKeyPair();
Key publicKey = kp.getPublic();
Key privateKey = kp.getPrivate();
public static void main(String[] args) {
 
 
   }

}

[+Asik]

So it's not. That line of code has to be inside a method, this is just a rule of the Java language. At the class level you can only have method or field declarations, and field declarations can optionally have an initializer. That line of code is neither.

[SPEhosting]

Dr_Asik, on 21 November 2012 - 22:26, said:

So it's not. That line of code has to be inside a method, this is just a rule of the Java language. At the class level you can only have method or field declarations, and field declarations can optionally have an initializer. That line of code is neither.

oooooh derp ... I did something stupid didnt I, I facepalmed when I realised I am not thinking at all .... (its been a while since i last slept)

[SPEhosting]

kinda want to get rid of my posts because i feel so stupid now ... can not believe I did such a stupid mistake im so angry with my self

[+theblazingangel]

Dr_Asik, on 21 November 2012 - 20:48, said:

best post in this entire forum since a long while.

thanks

SPEhosting, on 21 November 2012 - 09:24, said:

well I will answer the first two sub titles the 3rd one will require more reading but I have to get my washing out before i go to class xD ......

firstly thank you for taking the time to show interest it shows you are taking me seriously...

secondly This is my aim and how I am trying to get there...

this P2P connection will have a host so far im starting off easy, the host will be decided by the people using the chat (e.g. run host.class)

the host will be listening on the desired port (default probs 4444 or 5555 something like that) the client then connects to host ... yes target will need to be specified by client (I am trying to do this with as little database work as possible but I am not closed to it )

this program is not designed for use by a large populous its a program that will be off the radar for now and used for specific private communications....

I was thinking about using private and public key methods but im not sure how they work yet....

There will be another chat I will be making in the future which will use usernames and such both with encryption and without (one is a project I have to do other is just because I want to ) ....

like I said rest will have to wait!!! also no saved chats ever ! all will be burnt ...

Okay, small user base, we have a server, but no database so you've got no password protected user account mechanism, which imposes limits on what you can do. Let's see what we can come up with...

Users simply specify in their client the alias they wish to use. The client connects to the server, giving it the alias, which takes note of it and the IP address and port number of the computer registering it, and holds this information in memory with all of the others. When a chat client is started, it retrieves from the server a list of every user connected to the server, and periodically checks for updates. This list is presented to the user, from which they select someone to try and establish a chat session with.

Each user's client generates a key pair, and can save it (as opposed to generating a fresh one for each session). Public keys can be exchanged directly when establishing a chat session with someone. With the key saved and re-used the next time you use the chat client, this makes friend-list functionality viable. With friend list functionality, a copy of the friend's public key is saved. The next time you connect to them, from any IP address, or with any alias, the copy of your public key that they saved allows you to be recognised. To actually be highlighted as being online in another user's friend list, when you connect to the service, your certificate or at least the short identifier for it, should be uploaded to the server; This would then be distributed to other clients when they update their copy of the list of people connected to the server.

The key exchange between users needs to be verified though through other means. If verification is not done, there's no guarantee of any security what-so-ever. If users verifying each other's keys will be a pain, and it very likely will, you could use the 'web-service as a middle-man' approach that I described in my last post - create a key pair for the web service, and give a copy of the public key to everyone using that server, making sure they know they have got the correct one, and it hasn't been intercepted and switched (perhaps a big problem). Users would provide the web service's public key to their clients, which would use it to encrypt and send the user's public key to the server, which would de-crypt it, re-encrypt with the others user's public key, and pass it on to them. But as described in my last post, the admin of the web server could snoop! Users directly exchanging keys and verifying them is the correct and most secure way to do it, but a pita.

If a user needs to replace their key for any reason, the existing entry in someone else's friend list will be redundant. You would have to make contact with them, explain that you've changed your key, prove to them that it really is you, and do a verification of the new key. Unless you contact someone to get them to replace their friend list entry for you, there's no way that they would know that their existing one is redundant, they'll just think that you never come online any more.

I think that covers everything...

Oh, something I meant to talk about in my earlier post but forgot about - SPI, NAT & UPNP

If another chat client, or the web service, tries to send a communication of any kind to your client, that isn't a direct reply to a communication sent out by your client, then it's going to get blocked. To get around this, just like utorrent, your client is going to have to punch a hole through to the outside world with UPNP in order to allow all of the incoming traffic to pass through to your chat client.

[+articuno1au]

SPEhosting, on 22 November 2012 - 00:26, said:

kinda want to get rid of my posts because i feel so stupid now ... can not believe I did such a stupid mistake im so angry with my self

It's particularly amusing given you flamed off at someone and told them not to underestimate you

Also, I doubt this will end up having any relevance to your current work given how you've proceeded, but it's worth a read: http://en.wikipedia....an_key_exchange

It's a very cool idea and should help you with basic crypto things in the future.

Another "for extra reading" recommendation is "The Code Book" by Simon Singh. Very good place to start if you want a crash course in Cryptography

Good luck with your project >.<

[+theblazingangel]

articuno1au, on 22 November 2012 - 02:41, said:

Also, I doubt this will end up having any relevance to your current work given how you've proceeded, but it's worth a read: http://en.wikipedia....an_key_exchange

Very interesting!! A couple of possible issues using it here though:

• A piece of information needs to be shared between the two users. If a malicious person were to observe this piece of information being exchanged, that's not a problem, however if they were able to perform a man in the middle attack, then this would be useless. To eliminate the possibility of a man in the middle attack, you'd have to exchange it some other way, or at least confirm it some other way, i.e. over the phone.
  
• Additionally, if knowing that the person on the other end really is who they say they are, you still need the additional external verification.