• 0

Efficient/fast CSV file from client (Windows) to server (CentOS)


Question

Hello, I am seeking some guidance on the best possible way to send a CSV file from a Windows client to CentOS server with the lowest overheads. The CSV file changes perhaps 2-3 times a second and I need to send this to the server for data capture and processing. This send process happens for perhaps 8 hours and then sleeps for weeks or sometimes months. When not sleeping I need to send the CSV as fast as possible, once per second would be acceptable. Appreciate any suggestions or guidance.

 

Client side current: Run a program and sends data when datafile.csv file is modified. Uploads via WinSCP / SFTP. Overhead of SFTP mean transfer takes 1-3 seconds before sending latest data meaning might miss some critical data now and then. Over the course of ~8 hours I might miss say 10% of data.

Client side required: Run a program and sends current data CSV file to server as fast as possible. Ideally not a queue as queue would likely grow faster than file can be sent. Ideally executable file with no other software requirements. Ideal outcome would be data transfer of datafile.csv  from client/server started and finished once per second.

 

Server side : I am checking file timestamp and only running the software if datafile.csv is newer than 2 hours, after which it sleeps and checks every 1 minute following. I am happy with this.

 

CSV file : The file is small, ~100-200 bytes. Single line CSV with about 80 positions.

1 answer to this question

Recommended Posts

  • 0

Probably not going to get anything faster than a basic TCP connection. If you want to keep things as simple as possible, you could use a Powershell script to send the file  on the Windows client and Netcat on the Linux server to listen and receive the file.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • > Our goal is to ensure that the App Store remains an outstanding opportunity for developers and a safe, trusted experience for our users. There are so many scam apps on the platform that it is hard to believe they are truly interested in having a safe, trusted experience for their users.
    • Google announces upgraded Gemini 2.5 Pro model with enhanced capabilities by Pradeep Viswanathan Google today announced an updated Gemini 2.5 model with several improvements that can be observed in popular AI benchmarks. Google particularly highlighted that the new Gemini 2.5 Preview 06-05 "Thinking" model performs better in coding, math, science, and reasoning. Last month, during Google I/O, Google released an updated version of the Gemini 2.5 Pro model with significant improvements. Today's update builds on the release of the previous month with further enhancements. In addition to benchmark improvements, based on user feedback, this updated Gemini 2.5 Pro model also comes with improved style and structure, which will result in creative and better-formatted responses for end users. You can find the benchmark comparison with other leading AI models below. As you can notice in the above table, this updated Gemini 2.5 Pro preview model is now SOTA (State-of-the-Art) in coding benchmarks like Aider Polyglot. It also scored SOTA performance on GPQA and Humanity’s Last Exam (HLE) benchmarks, which test math, science, knowledge, and reasoning capabilities. In real-world testing, this latest 2.5 Pro model scored 24 points better on LMArena, maintaining its lead, and a 35-point jump on WebDevArena at 1443. Developers can access this latest Gemini 2.5 Pro preview model through the Gemini API via Google AI Studio and Vertex AI. General consumers will be able to access this model via the Gemini app. Google also confirmed that the Gemini 2.5 Pro model will be generally available in a couple of weeks, allowing developers to start using it in production-ready enterprise-scale applications.
    • The Bromance is definitely over! 🤣🤣🤣
    • I have the chance to live where electricity comes only from dams.
    • Which doesn't give them a pass for making the same game with a different skin
  • Recent Achievements

    • Week One Done
      jbatch earned a badge
      Week One Done
    • First Post
      Yianis earned a badge
      First Post
    • Rookie
      GTRoberts went up a rank
      Rookie
    • First Post
      James courage Tabla earned a badge
      First Post
    • Reacting Well
      James courage Tabla earned a badge
      Reacting Well
  • Popular Contributors

    1. 1
      +primortal
      411
    2. 2
      +FloatingFatMan
      181
    3. 3
      snowy owl
      177
    4. 4
      ATLien_0
      174
    5. 5
      Xenon
      135
  • Tell a friend

    Love Neowin? Tell a friend!