Creating a p2p network
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
torbente · 2014-01-06 09:38

Hi everyone;

I want create a program able to stablish p2p conections where users
can send and receive simple information (just strings, like "Hello my
name is Mike") between them using a secure conection (SSL maybe) Are
there any examples? Any orientation about how i can build this is very
grateful.

Since i dont have experience with networking development please be as
clear as you can.

----------------------------------------------------------------------
Ocye · 2014-01-06 11:56

Have you read the wiki page on Synapse? Perhaps you want to look as
well for "synapse chat example lazarus" with your preferred search
engine. As far as I know there is an example distributed with the
source code.

Quote from: torbente on January 06, 2014, 10:38:58 amSince i dont have
experience with networking development please be as clear as you can.
Please ask specific questions to get clear answers ;-).

----------------------------------------------------------------------
jarto · 2014-01-06 12:00

I'll try to write about the big picture. First you need to make a
server. You can do that using Indy's TTcpServer or Synapse's sockets.
Indy is bigger but it usually is an easier way to start. Synapse is
smaller but with it you need to do a lot more yourself.

Basically with both the idea is that the server creates a listening
socket that listens to port number you decide. When a connection to
that socket arrives, your server creates a new thread to handle that
connection. The thread itself will read and write to that socket. So,
if you have 10 connections to your server, you'll end up having 10
separate threads talking to those connections. Inside a thread you
basically then only read and write until the connection is closed.

A good way to start is to make the server first. You don't need to
make the client at the same time as you can test your server by using
telnet:

1. "telnet localhost your_port" should make your server create the
listening thread
2. write something and press enter -> your server should read what was
written
3. write to the connection from your server -> it should be shown in
the telnet window
4. close the connection from your server -> telnet should say that
connection closed

When the server works well, you can start writing your own client
software. It's basically just a simple tcp socket. Indy and Synapse
both have easy to use ones. In the client socket you specify host and
port and do a connect. And after that it can read and write to the
socket pretty much like the socket on the server side.

----------------------------------------------------------------------
torbente · 2014-01-06 13:21

ok, i downloaded indy package and installed it in lazarus v 1.0.14

I put a TIdTCPserver on my form and set values:
active: true
default port: 8080

when i run the program, my firewall told me that one program was
requesting autorization.

I wrote "telnet localhost 8080" and CMD witndow showed me "200
Welcome"
When i close the CMD window, my program told me that a connection was
closed gracefully.

I assume i was able to connect to the server.

I added a simple image which is visible when i connect and hides when
i disconnect. It works perfect 

Now i want that my programs show in the screen any text i write in the
telnet console; it minds, be able to read what the client "says".

----------------------------------------------------------------------
jarto · 2014-01-06 14:17

Great. That's progress :-)

Maybe these example can get you further:

http://sourceforge.net/projects/indy10clieservr/

----------------------------------------------------------------------
torbente · 2014-01-06 16:03

OMFG! Its working! 

I thing the worst part is over, i understand now how it works!

EDIT

Ok, i have a "stable" server now, compiled and able to accept multiple
client conexions.

Before try to make the conection secure (SSL implementation) i will
focus on try to know how it works in real life. I mind, with real IPs.
How the server and the client could know their own IP? Not netwrok IPs
(127.0.01) but internet ones.

Also, i dont have experience with p2p networks. To make it stable,
should i add one server and many client conecctions in the program?

I cant believe that in few hours i was able to do all this! Thanks a
lot!

EDIT 2: I was able to get the client local IP when it connects using:
Var
Newclient : string;
begin
   Newclient := AContext.Connection.Socket.Binding.PeerIP;
end;

But im only able to work using local IPs (IE 10.0.0.35) If i set the
client to connect to my external IP (ie 241.210.x.2 it did not work
(my server is open of course) How i can fix this?

Also, if i set the IP for my port := 241.210.x.2 i get an error that
address and port are already in use 

I checked the internet the last hours but i was unable to find nothing
to fix this.

----------------------------------------------------------------------
taazz · 2014-01-07 04:27

Is your external IP on a card on your working machine? is it on a
rooter if yes does it support NAT or port forwarding?

----------------------------------------------------------------------
torbente · 2014-01-07 04:47

I connect to internet via router but i dont have idea if it supports
NAT or port forwarding. 

EDIT: I need configure my router? It is not possible to configure it
directly from my aplication? I know there are many aplications that
allows p2p conections throught a router without change the
configuration of it.

EDIT 2: Mayve i need use a IdServerIOHandlerSSLOpenSSL1? Im in the
right way?

----------------------------------------------------------------------
taazz · 2014-01-07 05:12

those application and the router must support UPNP or NAT-PMP
protocols to auto setup the router there are no libraries tthat
implement those protocols that I know of.

----------------------------------------------------------------------
torbente · 2014-01-07 06:19

Well, im learning more aboyt networkig development in the last 24
hours that what i learned in all my life.

Can a "tunel" be created using lazarus? Any library? Im actually
trying with devart secure bridge.

----------------------------------------------------------------------
taazz · 2014-01-07 06:30

yes but with out UPNP or NATPMP you have the same problems with your
router's port forwarding.

----------------------------------------------------------------------
torbente · 2014-01-07 06:44

Quote from: taazz on January 07, 2014, 07:30:42 amyes but with out
UPNP or NATPMP you have the same problems with your router's port
forwarding.

I spent the last 15 years developing amateur stand-alone games,
UPNP/NATPMP is almost unreadable for me.
Secure bridge is not an option since is not freeware.
Can i have an example of lazarus program that created sucessfully a
tunel? I mean, i can use emule in my PC or any chat app (mIRC is a
good example) so this apps are able to stablish a tunel (i dotnt need
open ports manually in my router when i install it)

Maybe i must made a better explanation of what i want from my app:

A lot of people is connected p2p from their homes.

Tom
Mike
Billy
Peter
....

If Tom sends a string to the network, all the other users receive it
and knows who is sending it. Also, all users have a copy of all the
strings on their own machine. Something like a chat but without
personal messages (all strings are for all users)

Can it be implemented in lazarus to be used for everyone? Or it is
needed that everyuser reconfigures his router? (most users dont have
idea how to reconfigure router ports)

----------------------------------------------------------------------
taazz · 2014-01-07 06:58

Sorry I do not have any sample tunnel that I can share. As for the
rest google UPNP http://www.upnp.org/ and natpmp http://files.dns-
sd.org/draft-cheshire-nat-pmp.txtfor more information.

----------------------------------------------------------------------
torbente · 2014-01-07 07:30

Very sad, looks like i will not be able to create my app in lazarus 

If i reconfigure my ports and use my machine as a server, the clients
will need to reconfigure their router too?
I guess the clients will be able to send information to the server,
even if the server can not send nothing back.

----------------------------------------------------------------------
taazz · 2014-01-07 07:46

Yes clients will be able to send and receive information from a server
as long as the port the server uses is not blocked by their firewall.

----------------------------------------------------------------------
Giu · 2014-01-07 07:59

As. Taaz sais, you must open port in router and redirect to your local
ip. You just need to do this, where connection should be established
(ex: where server resides  not clients)

But this problem, you have it with Lazarus and whatever language you
use to develop your application. It's not a Lazarus problem, it's a
"network" problem.

You can avoid NAT, using an intermediate application. Just like
Teamviewer for example does. Teamviewer don't need NAT because all
communications first goes by its servers.

For this, you need a "bridge" app, communicating with all clients. I
mean. Imagine I want to connect to your server. Your server is
connected to the bridge, waiting commands. My client connect to the
"bridge app", and I tell it, I want to connect to your server. The 
"bridge " then pass  communications to your server, and send and
receive all communications.  This way, you don't need to do UPNP, NAT
and so on, but needs a public server for communication between clients
purposes .

I hope I explained correctly.

EDITES:::
QuoteIf i reconfigure my ports and use my machine as a server, the
clients will need to reconfigure their router too?
No. You just need to reconfigure router where connection "destination"
needs to be established. Where server is.

When you try to send a communication on a internet IP, normally  you
find a router. What NAT does, is to tell router what to do with the
packets received under some ports. If this is not configured  routers
don't know what to do with this information  because you can have
multiple clients connected to a router, and router can't send all this
information to all clients. With NAT you say to router.."hey,
communication from port 2345 should go to local ip 192.168.1.3".

----------------------------------------------------------------------
zeljko · 2014-01-07 08:10

Quote from: torbente on January 07, 2014, 07:19:01 amWell, im learning
more aboyt networkig development in the last 24 hours that what i
learned in all my life.

Can a "tunel" be created using lazarus? Any library? Im actually
trying with devart secure bridge.

Yes you can create remote "tunnel" by using libssh2 (I'm using such
solution for my apps on linux and mac, haven't tried on windows, but
I'm sure that it'll work if libssh2.dll exists) (like ssh with -R
option).

----------------------------------------------------------------------
torbente · 2014-01-07 19:00

Ok, lets me see if i understood.

- If i set my IP to work as a server, i can distribute my app so
others can connect as clients, right?

If this is correct, i could design the app to be able to work in 2
diferent ways, as a server or client, depending of what the users
decides.

Lets make this readable:

- My program starts
- Connects to a known server
- The server knows my external IP and send it to me.
- I try to start my own server; if i had the port redirected, it will
works. Now, i can receive incoming conexions meanwhile im still
connected to another server. If i cant start my own server im still
conected to the network as client.

Is this idea ok? Or did i miss something?

Another thing: AFAIK the cumber of incoming connections is... acording
to my system limits. Can someone tell me a reasonable number to do not
freeze my PC?

Thanks to you all guys for your time.

----------------------------------------------------------------------
jarto · 2014-01-07 19:53

Quote from: torbente on January 07, 2014, 08:00:46 pm- My program
starts
- Connects to a known server
- The server knows my external IP and send it to me.
- I try to start my own server; if i had the port redirected, it will
works. Now, i can receive incoming conexions meanwhile im still
connected to another server. If i cant start my own server im still
conected to the network as client.

Sure, you can do all that with Lazarus. If you can't do it with
Lazarus, you can't do it with any other programming language.

If you have problems to connect from an external ip to your server
program, it can be because of these reasons:

1. The operating system itself may want to prevent you from creating
listening sockets. For example, in Linux that usually requires
superuser rights. If you don't have the necessary rights, there will
be an error when you try to create that listening socket in your code.

2. Some other program may already be listening to the same port. For
example, you may have a web server listening on port 80. If so, your
program can't listen on the same port at the same time.

3. Firewall. Even if you create a program that acts as a server, you
need to tell the firewall to allow external computers to connect to
it. Because your computer is probably behind a NAT firewall, you need
to tell the router to allow the connection and to forward it to a
certain port on your computer. Let's assume that your external ip is
112.221.123.121 and your computer's ip-address behind the router is
192.168.0.30 and you listen on port 6610. Then you need to create a
NAT rule on your router to forward all traffic from port 6610 to
192.168.0.30:6610.

If you want to create a program that changes those firewall rules
itself, well, don't bother thinking about it. The firewalls exist for
a reason and they are meant to be hard to circumvent. The best you can
do is to instruct your users to create the rules themselves if they
want to use your program.

And to be clear, the firewall usually isn't that picky about outgoing
connections (your computer accessing a service on another computer on
the internet). It's picky about external computers trying to access a
service on your computer.

----------------------------------------------------------------------
Giu · 2014-01-07 20:06

Quote from: torbente on January 07, 2014, 08:00:46 pmOk, lets me see
if i understood.

- If i set my IP to work as a server, i can distribute my app so
others can connect as clients, right?
Yes.

QuoteIf this is correct, i could design the app to be able to work in
2 diferent ways, as a server or client, depending of what the users
decides.
Of course, this is your decission. You can parametrize this from UI or
with a INI file for example.

QuoteLets make this readable:

- My program starts
- Connects to a known server
- The server knows my external IP and send it to me.
- I try to start my own server; if i had the port redirected, it will
works. Now, i can receive incoming conexions meanwhile im still
connected to another server. If i cant start my own server im still
conected to the network as client.

Is this idea ok? Or did i miss something?

I lost in 3rd step. Why the client connects to another server instead
to connect just to you? If you have another server, connect all
clients to the same. You don't need to act as a server too.

QuoteAnother thing: AFAIK the cumber of incoming connections is...
acording to my system limits. Can someone tell me a reasonable number
to do not freeze my PC?

Thanks to you all guys for your time.

For a simple "internal" chat application, I think you don't have to
worry about system limits.

----------------------------------------------------------------------
arneolav · 2014-01-07 21:51

This program has been very useful fore me: PFPortChecker

Just key in the actual Ip adress and portnumber.
In a few seconds the Poerchecker tells you if the port is open or not.

----------------------------------------------------------------------
torbente · 2014-01-07 22:28

ok, i entered in my router config and i found the "port forwarding"
section

Port Forwarding:  Disable    Enable     
Protocol:    Comment:      Enable
Local IP Address:       Local Port:  -  
Remote IP Address:     Public Port: - 
               Interface:  

i enabled it
protocol: both-tcp-UDP  I selected TCP
localipadress: 10.0.0.35
local port: 6000-6001
remotetipadress - blank
Public port - blank
Interface Any-VcO I selected Any

I start the server on my machine, using 10.0.0.35:6000
Then i start the client from my machine too, conecting to my public IP
121.xxx.xxx.2:6000

I receive the message error: Socket error 10061 Conection refused

What i did wrong?

----------------------------------------------------------------------
Giu · 2014-01-07 23:13

Quote from: torbente on January 07, 2014, 11:28:11 pmok, i entered in
my router config and i found the "port forwarding" section

Port Forwarding:  Disable    Enable     
Protocol:    Comment:      Enable
Local IP Address:       Local Port:  -  
Remote IP Address:     Public Port: - 
               Interface:  

i enabled it
protocol: both-tcp-UDP  I selected TCP
localipadress: 10.0.0.35
local port: 6000-6001
remotetipadress - blank
Public port - blank
Interface Any-VcO I selected Any

I start the server on my machine, using 10.0.0.35:6000
Then i start the client from my machine too, conecting to my public IP
121.xxx.xxx.2:6000

I receive the message error: Socket error 10061 Conection refused

What i did wrong?

- You can't connect to itself using you public IP if your router
doesn't accept loopback connections.
- You must specify public port. You can specify both, to NAT port too.
Example: I'm running an Apache instance in port 8080 on my computer,
and I want to expose to public, but public port will be 81. Then, all
connections to my router from outside (PUBLIC port) from 81 port, will
be NAT to my local IP address at port 8080. You get the point?

Attached there are an example of my router. My router accept
connections to port 35001, and redirects to 192.168.1.11 on port 35001
(private) too.

Regards.

----------------------------------------------------------------------
torbente · 2014-01-08 03:40

I  cant configure the port forwarding in my router, i feel like a
dummie. (sceenshot attachment) I remember that 10 years ago or more, i
did something like this in my old router to be able to play Age of
Empires 

What else could be?

How i can intercept runtime errors? I think i remember something like
IOresult or something like that in turbo pascal 7, but it was many
years ago.

Please, help me with this issue. Im working in other aspects of the
app meanwhile im able to fix this.

----------------------------------------------------------------------
taazz · 2014-01-08 04:11

why don't you do your own research?
http://portforward.com/english/routers/port_forwarding/routerindex.htm
it took me 2 minutes to get the above link. In your case leave the
external IP empty and try to set the interface to the card/interface
that connects to the internet.

----------------------------------------------------------------------
torbente · 2014-01-08 04:33

Sorry if you think im a lazy guy   Im constantly searching the web
before and after i post here. My router is a Ryge WA41R but i cant
find a single link referred to how port forward it. The link you just
posted  do not contains nothing usefull.
I tried to configure the port forwarding for hours, until my friend
(who where runing the client from his own home) get tired and went to
the bed.
Im actually trying to figure how i can intercept errors and how i can
obtain the local IP. So i think will be very usefull if i keep the
thread ordered:

Looking for:
- How i can get the local IP using indy.
- How i can manage errors using indy (for example, when i call
TCPServer.connect and there are no host, i want be able to show a
message "Unable to connect" in the memo)

Solved:
- Create a client and a server for simple text exchange using Indy.

I will share the code for a server&client app when im able to do it.

----------------------------------------------------------------------
taazz · 2014-01-08 05:21

Quote from: torbente on January 08, 2014, 05:33:28 amSorry if you
think im a lazy guy   Im constantly searching the web before and after
i post here. My router is a Ryge WA41R but i cant find a single link
referred to how port forward it. The link you just posted  do not
contains nothing usefull.
I tried to configure the port forwarding for hours, until my friend
(who where runing the client from his own home) get tired and went to
the bed.
Im actually trying to figure how i can intercept errors and how i can
obtain the local IP. So i think will be very usefull if i keep the
thread ordered:

Looking for:
- How i can get the local IP using indy.

http://stackoverflow.com/questions/15071564/how-to-get-local-
ip4-address-using-indy

Quote from: torbente on January 08, 2014, 05:33:28 am
- How i can manage errors using indy (for example, when i call
TCPServer.connect and there are no host, i want be able to show a
message "Unable to connect" in the memo)

its a sum of exceptions and events fired eg onstatus onconnect etc. so
using something along the lines ofCode: [Select]  try
    IdTCPClient1.Connect;
  except
    on E : EIdConnectTimeout do begin
      // server did not respond might be down or out of reach.
    end;
    on E:EIdException do begin
      //general Indy exception handler
    end;
  end;

----------------------------------------------------------------------
torbente · 2014-01-08 10:05

I received an error, "GStack" and "IPv4ToDWord" as not found
identifiers. I supposse i need include the required unit in "uses",
but i dont know which one is required 

As for error management, indy handles any error like an exception? Are
there a complete list of posible errors listed ?

EDIT: Sorry if i have too many questions

Im trying to call the procedure to start the TCP server from another
form(unit), but i cant find the way since the procedure is
tform2.startserver and i dont know how i could make it public. If i
call "startserver" or "tform2.startserver" dont work. Any idea?

----------------------------------------------------------------------
taazz · 2014-01-08 11:32

grep (aka search in files) the indy folders to find the units that use
or declare the missing keywords.

----------------------------------------------------------------------
JD · 2014-01-08 12:18

"IPv4ToDWord" error ------> add IdGlobal to your uses clause
"GStack" error            ------> add IdStack to your uses clause

All Indy exceptions are of the type EIdXXXXXXX.

JD

----------------------------------------------------------------------
torbente · 2014-01-08 12:59

Thanks JD, very usefull answer 

Now i get this error

unit2.pas(48,12) Error: identifier idents no member
"AddLocalAddressesToList"

In other hand, i know i can handle all errors here

Code: [Select] 
procedure TForm2.TCPServerException(AContext: TIdContext; AException:
Exception);
begin
   {if Aexception = EIDSocketError then halt(0); }
end; 
But i dont know how exactly  One clear example could be very helpfull

EDIT: I just realized that the app need know if the port is redirected
to start working as a server   

EDIT2: If you are a master in lazarus+indy+ssl protocol, feel free to
PM me so we can talk via skype

EDIT3 Looking Inet library to see if it could work better for me.

EDIT 4 I Installed Inet. Looks like this library contains all i need
I will keep you info as im using testnet example as a guide

----------------------------------------------------------------------
torbente · 2014-01-11 10:42

Please, feel free to close this thread. I finally got the way. I will
post the final code soon.

Thanks to you all guys for your help

----------------------------------------------------------------------
Delphier · 2020-01-22 00:27

Have you found a solution ?
would you please share last results

best regards