Lightweight Network Statistic Programs

[BinaryData]

Hey Guys,

 

I'm looking for a lightweight bandwidth / network statistics program. Something that could run on a Raspberry Pi 1/2/3 or something of the sort. I've seen Suricata, and a few others but they are all resource intensive. I'm hoping to find something that's as simple as logging how much bandwidth is used in a single day, week, and month.

 

I've looked into coding but my skills aren't up to that level.

[FiB3R]

https://pi-hole.net

 

Enough info?

2 birds with 1 stone?

[BinaryData]

1 minute ago, FiB3R said:

https://pi-hole.net

 

Enough info?

2 birds with 1 stone?

We will see. Let's see if it works on my digital oceans server.

[+BudMan MVC]

pi-hole is just a dns blocker of ads.. Not sure how that would give you how much traffic moving?  It will show you how many ads you blocked sure.

 

"logging how much bandwidth is used in a single day, week, and month."

 

Of the pi, some other box?  http://humdi.net/vnstat/

 

Is really light.. What exactly are you wanting to monitor, are you talking about your seedbox?  That is going to be a bit difficult since its shared and your not root.  If your talking about stuff on your local network, or a vps then there are plenty of options..

[BinaryData]

1 hour ago, BudMan said:

pi-hole is just a dns blocker of ads.. Not sure how that would give you how much traffic moving?  It will show you how many ads you blocked sure.

 

"logging how much bandwidth is used in a single day, week, and month."

 

Of the pi, some other box?  http://humdi.net/vnstat/

 

Is really light.. What exactly are you wanting to monitor, are you talking about your seedbox?  That is going to be a bit difficult since its shared and your not root.  If your talking about stuff on your local network, or a vps then there are plenty of options..

Nah, it's for a Digital Ocean's server I've got. Since my work complains about using personal computers at work (SSH), I use a $5 digital ocean server for projects. They've got a 1TB limit and I'm running various things on it, just to brush up on things. I can nuke it if I mess things up, takes 30s to wipe it and reload. The seedbox isn't really a bother anymore, I'm pushing 10 - 20TB of seeded content every month. Highest we've hit was 21.7TB of bandwidth. 

[+BudMan MVC]

Ah so its a vps, yeah just install vnstat, there is even a gui for it.  You can get exactly what your looking for. 

 

What does that have to do with a pi??  You would install such tools on the box itself.. Especially when its out in the cloud somewhere, not on your own network.

[BinaryData]

3 minutes ago, BudMan said:

Ah so its a vps, yeah just install vnstat, there is even a gui for it.  You can get exactly what your looking for. 

 

What does that have to do with a pi??  You would install such tools on the box itself.. Especially when its out in the cloud somewhere, not on your own network.

If the Pi could run the software, then my vps could run it. It's 1 Processor with 512MB Memory, and 30GB of space. Just enough to run the things I need. vnstat is pretty much what I was looking for. I saw MRTG, but I wasn't sure how resource intensive it was.

[BinaryData]

And it's soooo much simpler to do sudo apt-get install vnstat. No compiling needed, fark my life.

 

Awesome. I'm loving this little application already. I'll be adding it to one of my projects at home. Thanks again BudMan!

[+BudMan MVC]

mrtg is not all that cpu heavy, its just a bit complicated to setup..  And its not going to even give you want you want, sure it can graph interface usage.  But its not going to give you a breakdown of daily, weekly, monthly stats.  I would go with vnstat

 

simple

apt-get install vnstati and there you go..  Then you can edit its database to only monitor your actual interface your interested in.  VPSes quite often have lots of interfaces that are not really your actual traffic and will just be noise..  All depends on how your host has them setup.

 

Here is Just installed on one of my vps

 

 

	root@ns1:/# vnstat
	Database updated: Fri May 27 15:19:24 2016
	   venet0 since 05/27/16
	          rx:  381 KiB      tx:  420 KiB      total:  801 KiB
	   monthly
	                     rx      |     tx      |    total    |   avg. rate
	     ------------------------+-------------+-------------+---------------
	       May '16       381 KiB |     420 KiB |     801 KiB |    0.00 kbit/s
	     ------------------------+-------------+-------------+---------------
	     estimated        --     |      --     |      --     |
	   daily
	                     rx      |     tx      |    total    |   avg. rate
	     ------------------------+-------------+-------------+---------------
	         today       381 KiB |     420 KiB |     801 KiB |    0.12 kbit/s
	     ------------------------+-------------+-------------+---------------
	     estimated        --     |      --     |      --     |
	root@ns1:/#
	

vnstati will let you create pretty graphs when and iff you want.  Or you could install the web frontend, but then your going to need a httpd, php, etc.  Not sure what you have installed on the box and if trying to keep the footprint small, etc.  If you wanted you could just create a cron for vnstati to spit out some pretty picks

 

If you want real time output there are plenty of other tools as well. iptraf, ntop, bmon, there is a ###### load of them..

 

[BinaryData]

root@<redacted>:~#        May '16      2.26 MiB  /     195 KiB  /    2.45 MiB  /       0 KiB
>          today      2.26 MiB  /     195 KiB  /    2.45 MiB  /      --
>
>  eth1: Not enough data available yet.
>
 

I have vnstat and vnstati installed right now. I have 2 eths, 0 is what's displayed, and 1 hasn't had enough data.

[+BudMan MVC]

maybe nothing is flowing over that interface.  Vnstat just runs a cron that grabs the stats off the interface and presents it an easy to view manner.

 

If your not using eth1 then just don't have vnstat look at it.  Like I said my vps has loads of interfaces listed, gre greTap0, as0tap0 aso0tap1, etc.  Which is how the thing is setup  - but the traffic that is what the box is moving to and from the net is on venet0, there are also some sub interfaces to that.  Like mentioned before just clean up the interfaces that you have vnstat monitoring..

[BinaryData]

10 hours ago, BudMan said:

maybe nothing is flowing over that interface.  Vnstat just runs a cron that grabs the stats off the interface and presents it an easy to view manner.

 

If your not using eth1 then just don't have vnstat look at it.  Like I said my vps has loads of interfaces listed, gre greTap0, as0tap0 aso0tap1, etc.  Which is how the thing is setup  - but the traffic that is what the box is moving to and from the net is on venet0, there are also some sub interfaces to that.  Like mentioned before just clean up the interfaces that you have vnstat monitoring..

Well, I've got it running on the server I needed it on. The problem with Digital Ocean is that it won't handle what I'm throwing at it.

 

I've got it setup on my linux box at home, and it's already pulling good stats. I'll end up installing MRTG to export fancy images of network traffic, unless you can suggest something else.

[+BudMan MVC]

Handle what? You have what a 5$/month vps there?  Pretty sure it can run vnstat

 

Not sure what kind of fancy pictures you want.. Running real time graph?  They don't provide you with that? Then get a vps somewhere that does.

 

Is something like this what your looking for? I can send link to company, I have a vps there..

 

I don't do much with this one, its a ssh server for me is all in the florida area of the US.. Most of my vps sit idle until I need something.. So they could go weeks without me even logging in, etc.

 

You do understand vnstati can dump out some pretty pictures of your network usage..  As to mrtg, yes very nice product - can graph all kinds of stuff.. Here is a graph setup of my ntp server in a few minutes...  But I highly doubt this is "pretty" enough for you

 

Observium has some pretty stuff, which I would show you but looks like my freaking SD on that pi bit the dust.. Have to rebuild that pi.. Back up your SD images people   even on your play stuff hehe.. Good excuse to use the latest version of raspian..

 

[BinaryData]

1 minute ago, BudMan said:

Handle what? You have what a 5$/month vps there?  Pretty sure it can run vnstat

 

Not sure what kind of fancy pictures you want.. Running real time graph?  They don't provide you with that? Then get a vps somewhere that does.

 

Is something like this what your looking for? I can send link to company, I have a vps there..

 

I don't do much with this one, its a ssh server for me is all in the florida area of the US.. Most of my vps sit idle until I need something.. So they could go weeks without me even logging in, etc.

 

You do understand vnstati can dump out some pretty pictures of your network usage..  As to mrtg, yes very nice product - can graph all kinds of stuff.. Here is a graph setup of my ntp server in a few minutes...  But I highly doubt this is "pretty" enough for you

 

Observium has some pretty stuff, which I would show you but looks like my freaking SD on that pi bit the dust.. Have to rebuild that pi.. Back up your SD images people   even on your play stuff hehe.. Good excuse to use the latest version of raspian..

 

MRTG is fine, lol. It's just going to be linked to a webpage for statistics purposes. Mostly wanted to show;

 

Incoming:

Outgoing:
Weekly Summary:

Montly Summary:

Overall Summary:

 

Which, mrtg / vnstati does for the most part. I was trying to figure out how to get vnstati to output to a web directory, but I figured out how to do that with mrtg, lol. It outputs a .png, which can be linked http://localhost/imgs/mrtg_png_file.png or whatever you want.

 

I did look at the Tx/Rx of my router since I got it. Pretty sure I can't count that high, haha!

[+BudMan MVC]

If you want fancy looking graphs.. I like the info I pull from my UPS

 

 

This is using PRTG -- that for sure I know you could setup, since its click click   and it does auto discovery on most things..

 

As to getting vnstati to dump a pic

 

vnstati -s -i eth0 -o /tmp/vnstat.png

Output traffic summary for interface eth0 to file /tmp/vnstat.png.

 

Really - you can not figure out how to post it to your www dir??

 

Here just dumped this from the vps I installed it on yesterday

 

 

[BinaryData]

I'm forcing myself to use CLI instead of GUI because of my up coming Linux+ classes. Becoming more familiar with the CLI is a whole lot better. Granted the GUI is WAY easier for me, but knowing my way around the CLI is good for SSH as well. It means I can stop being a pansy and run Server version instead of Desktop, which I hate using. With enough time and dedication, I could figure it out. However, for some reason my work is blocking 80% of the sites I go too. I had to specifically request them to unblock Neowin, I explained that it was an educational site, and a tech news related site. Hell they even blocked askubuntu. I submitted an excel doc with 120+ links to unblock. It's like they clicked deny-all on the damn firewalls.

[BinaryData]

I'm looking into setting up a cronjob to export that png gfx to a webserver to have it auto displayed every x hours

 

24 hours / Weekly / Monthly / Total

 

That's the graphics I'll have setup. Pretty sure vnstat does that from what I've seen.

[+BudMan MVC]

yup its that simple

 

Why not have the cron update the pics every 5 minutes?  Could all be put 1 script you call to create your graphs from 1 cron job.  That way all your graphs are pretty close to real time.

 

Just want to make sure your vnstat cron that collects the info runs before your output of the gfx, or you could end up out of sync and say 10 minutes off with your data, etc.

[BinaryData]

1 minute ago, BudMan said:

yup its that simple

 

Why not have the cron update the pics every 5 minutes?  Could all be put 1 script you call to create your graphs from 1 cron job.  That way all your graphs are pretty close to real time.

Wasn't sure how often I should set the generation, but 5 minutes works. Now, tonight at work, I hope I have nothing to do so I can setup cronjobs

[+BudMan MVC]

see my edit, you want to make sure the order your jobs will run in, because I do believe the default for vnstat to takes it stats is 5 minutes.  Maybe its 1?  Would have to look.  But if you have say your gfx job kick off at the same time vnstat is taking its stats you could end up with data that is not quite true or farther behind than you think it is, etc.

 

So you want to run your vnstat cron, make sure its done and then run your gfx output command.

[BinaryData]

Well, if it runs every 5 minutes, I could set it for 6 minutes and be in the clear. When I get the ESXi Box up and going, I figure I would setup PRTG, hell if it was "affordable" I would buy it. I don't get the whole sensors thing, but I haven't fully read the webpage. I'll do that tonight when I bbq my steaks. Bout as pretty as a graph I need!

 

I'll work on that cronjob some tonight, before I run it, I'll post it here for pointers, and potential flaws in it. Shouldn't be too hard.

 

[+BudMan MVC]

so the sensors is how many tests you have running.  Each test is a sensor.

 

So for example if you ping a host - 1 sensor, if you check that this host answers http, 2nd sensor.  If you free space on C drive another sensor..

[BinaryData]

On 5/31/2016 at 11:19 AM, BudMan said:

so the sensors is how many tests you have running.  Each test is a sensor.

 

So for example if you ping a host - 1 sensor, if you check that this host answers http, 2nd sensor.  If you free space on C drive another sensor..

I installed it. Rather interesting application. From the looks of the default install, I've got 26 sensors.

 

I did notice something, maybe it's something I haven't searched around enough for, but it doesn't pick up WiFi devices.

 

Though, I could see why not, each cell phone connected would be a sensor, and I could see how that could balloon out of control. I'll keep clicking on things, I think I need a VM specifically for this, shame it costs so much for a subscription. I emailed about an educational discount, wonder what the response will be.