Efficient/fast CSV file from client (Windows) to server (CentOS)

[sini]

Hello, I am seeking some guidance on the best possible way to send a CSV file from a Windows client to CentOS server with the lowest overheads. The CSV file changes perhaps 2-3 times a second and I need to send this to the server for data capture and processing. This send process happens for perhaps 8 hours and then sleeps for weeks or sometimes months. When not sleeping I need to send the CSV as fast as possible, once per second would be acceptable. Appreciate any suggestions or guidance.

 

Client side current: Run a program and sends data when datafile.csv file is modified. Uploads via WinSCP / SFTP. Overhead of SFTP mean transfer takes 1-3 seconds before sending latest data meaning might miss some critical data now and then. Over the course of ~8 hours I might miss say 10% of data.

Client side required: Run a program and sends current data CSV file to server as fast as possible. Ideally not a queue as queue would likely grow faster than file can be sent. Ideally executable file with no other software requirements. Ideal outcome would be data transfer of datafile.csv  from client/server started and finished once per second.

 

Server side : I am checking file timestamp and only running the software if datafile.csv is newer than 2 hours, after which it sleeps and checks every 1 minute following. I am happy with this.

 

CSV file : The file is small, ~100-200 bytes. Single line CSV with about 80 positions.

[+virtorio MVC]

Probably not going to get anything faster than a basic TCP connection. If you want to keep things as simple as possible, you could use a Powershell script to send the file  on the Windows client and Netcat on the Linux server to listen and receive the file.