A "Copythese" app I created with chatGPT.

[+Warwagon MVC]

At my GF's house, I have a large external hard drive I use as an offsite backup with many media files on it.

I have a connection from a headless computer at her house behind her TV which connects to my VPN which lets me have access to the drive, but when I put new things on it, it can amount to 50GB+ of data and up until recently she did not have unlimited internet, otherwise i would have just used something like syncback and synced the files over the internet across the VPN.

So I wanted an app, that could look at 2 directories and save the list of missing files to a TXT file, then I could point the program at home to the source directory (The directory that has the files the drive at her house needs) to a destination directory (an external hard drive I could put the files on to take to her house)

I then load the needthese.txt file created earlier with a list of all the files and hit copy and it copies just those files

So I went to Chat GPT for help and worked with it over the course of 8 hours to create an app that works great. The program checks the file size of each file before the copy and after it copies the file to make sure they are the same size.

After the transfer, it gives a list of files it transferred and a list of files it couldn't locate or files that it skipped because it was already on the destination.

it will not overwrite anything. If the file already exists it just skips it completely and puts it in the summary that it already existed in the destination.

I should note, per the source, that the app does ignore .nfo. .sub and .srt files.

Demonstration on how it works

This is what the contents of the needthese.txt looks like.

Here is the source code

import os
import wx
import wx.lib.filebrowsebutton as filebrowse
import threading
import shutil

# Global variables
source_directory = ""
destination_directory = ""
files_to_copy = []
copied_files = []
skipped_files = []
not_found_files = []
total_files_to_copy = 0
progress_percent = 0  # Progress for both copy and move operations
move_files = False

# Extensions to ignore
ignore_extensions = {".nfo", ".sub", ".srt"}


class FolderComparisonTool(wx.Frame):

    def __init__(self, parent, id, title):
        wx.Frame.__init__(self, parent, id, title, size=(600, 400))

        # Create a panel
        panel = wx.Panel(self)

        # Create and set up the source and destination file browse buttons
        self.source_browse = filebrowse.DirBrowseButton(panel, -1, size=(500, -1), changeCallback=self.on_source_select)
        self.source_browse.SetValue("")
        self.destination_browse = filebrowse.DirBrowseButton(panel, -1, size=(500, -1), changeCallback=self.on_destination_select)
        self.destination_browse.SetValue("")

        # Create the "Load needthese.txt" and "Create needthese.txt" buttons
        load_needthese_button = wx.Button(panel, label="Load needthese.txt")
        create_needthese_button = wx.Button(panel, label="Create needthese.txt")

        # Bind button events to their respective methods
        self.Bind(wx.EVT_BUTTON, self.load_needthese_txt, load_needthese_button)
        self.Bind(wx.EVT_BUTTON, self.create_needthese_txt, create_needthese_button)

        # Create a checkbox for moving files
        self.move_files_checkbox = wx.CheckBox(panel, label="Move files instead of copying")
        self.move_files_checkbox.Bind(wx.EVT_CHECKBOX, self.toggle_move_files)

        # Create a "Start Copying" button
        compare_button = wx.Button(panel, label="Start Copying")
        compare_button.Bind(wx.EVT_BUTTON, self.compare_folders)

        # Create a progress bar
        self.progress_bar = wx.Gauge(panel, -1, range=100, size=(500, -1))

        # Create labels
        self.files_left_label = wx.StaticText(panel, label="Files left to copy: 0")
        self.current_file_label = wx.StaticText(panel, label="Copying: -")

        # Set up the layout using a vertical box sizer
        sizer = wx.BoxSizer(wx.VERTICAL)
        sizer.Add(wx.StaticText(panel, label="Source Directory:"), 0, wx.ALL, 5)
        sizer.Add(self.source_browse, 0, wx.ALL | wx.EXPAND, 5)
        sizer.Add(wx.StaticText(panel, label="Destination Directory:"), 0, wx.ALL, 5)
        sizer.Add(self.destination_browse, 0, wx.ALL | wx.EXPAND, 5)
        sizer.Add(load_needthese_button, 0, wx.ALL, 5)
        sizer.Add(create_needthese_button, 0, wx.ALL, 5)
        sizer.Add(self.move_files_checkbox, 0, wx.ALL, 5)
        sizer.Add(compare_button, 0, wx.ALL, 5)
        sizer.Add(self.progress_bar, 0, wx.ALL | wx.EXPAND, 5)
        sizer.Add(self.files_left_label, 0, wx.ALL, 5)
        sizer.Add(self.current_file_label, 0, wx.ALL, 5)
        panel.SetSizer(sizer)

        # Initialize the wx event loop
        self.Show(True)

    def on_source_select(self, event):
        global source_directory
        source_directory = self.source_browse.GetValue()

    def on_destination_select(self, event):
        global destination_directory
        destination_directory = self.destination_browse.GetValue()

    def toggle_move_files(self, event):
        global move_files
        move_files = self.move_files_checkbox.GetValue()

    def compare_folders(self, event):
        global source_directory, destination_directory, files_to_copy, copied_files, skipped_files, not_found_files, total_files_to_copy

        source_directory = self.source_browse.GetValue()
        destination_directory = self.destination_browse.GetValue()

        if not source_directory or not destination_directory:
            wx.MessageBox("Please select Source and Destination Directories.", "Folder Error", wx.OK | wx.ICON_WARNING)
            return

        if not files_to_copy:
            wx.MessageBox("No files found in needthese.txt.", "No Files to Copy", wx.OK | wx.ICON_INFORMATION)
            return

        copied_files = []
        skipped_files = []
        not_found_files = []
        total_files_to_copy = len(files_to_copy)
        self.update_progress_bar(0)
        self.files_left_label.SetLabel(f"Files left to copy: {total_files_to_copy}")
        threading.Thread(target=self.copy_files).start()

    def copy_files(self):
        global source_directory, destination_directory, files_to_copy, copied_files, skipped_files, not_found_files, move_files, progress_percent

        size_mismatch_files = []

        while files_to_copy:
            file_info = files_to_copy[0]
            file_number = file_info[0]
            file_name_to_copy = file_info[1]

            try:
                source_file_path = self.find_file_in_directory(source_directory, file_name_to_copy)
                if source_file_path:
                    destination_file_path = os.path.join(destination_directory, file_name_to_copy)

                    if os.path.exists(destination_file_path):
                        skipped_files.append(file_name_to_copy)
                    else:
                        self.current_file_label.SetLabel(f"Copying: {file_name_to_copy}")
                        if move_files:
                            self.move_file(source_file_path, destination_file_path, file_name_to_copy)
                        else:
                            original_size = os.path.getsize(source_file_path)
                            self.copy_file_with_progress(source_file_path, destination_file_path, file_name_to_copy)
                            copied_size = os.path.getsize(destination_file_path)

                            if original_size != copied_size:
                                size_mismatch_files.append(file_name_to_copy)

                        copied_files.append(file_name_to_copy)

                    files_to_copy.pop(0)
                    self.files_left_label.SetLabel(f"Files left to copy: {len(files_to_copy)}")
                    self.current_file_label.SetLabel("Copying: -")
                else:
                    not_found_files.append(file_name_to_copy)
                    files_to_copy.pop(0)
                    self.files_left_label.SetLabel(f"Files left to copy: {len(files_to_copy)}")
            except Exception as e:
                wx.MessageBox(f"An error occurred while copying {file_name_to_copy}: {str(e)}", "Copy Error", wx.OK | wx.ICON_ERROR)

        if size_mismatch_files:
            self.show_copy_summary(size_mismatch_files)
        else:
            self.show_copy_summary()

    def copy_file_with_progress(self, source, destination, file_name):
        try:
            file_size = os.path.getsize(source)
            copied_size = 0
            buffer_size = 1024 * 1024

            with open(source, 'rb') as src_file, open(destination, 'wb') as dest_file:
                while True:
                    buffer = src_file.read(buffer_size)
                    if not buffer:
                        break
                    dest_file.write(buffer)
                    copied_size += len(buffer)
                    progress_percent = int((copied_size / file_size) * 100)
                    self.update_progress_bar(progress_percent)
        except Exception as e:
            wx.MessageBox(f"An error occurred while copying {file_name}: {str(e)}", "Copy Error", wx.OK | wx.ICON_ERROR)

    def move_file(self, source, destination, file_name):
        try:
            shutil.move(source, destination)
            progress_percent = 100
            self.update_progress_bar(progress_percent)
        except Exception as e:
            wx.MessageBox(f"An error occurred while moving {file_name}: {str(e)}", "Move Error", wx.OK | wx.ICON_ERROR)

    def update_progress_bar(self, percent):
        self.progress_bar.SetValue(percent)

    def find_file_in_directory(self, directory, filename):
        for root, _, files in os.walk(directory):
            if filename in files:
                return os.path.join(root, filename)
        return None

    def load_needthese_txt(self, event):
        global source_directory, files_to_copy

        source_directory = self.source_browse.GetValue()

        if not source_directory:
            wx.MessageBox("Please select the Source Directory.", "Folder Error", wx.OK | wx.ICON_WARNING)
            return []

        with wx.FileDialog(self, "Load needthese.txt", wildcard="Text Files (*.txt)|*.txt",
                           style=wx.FD_OPEN | wx.FD_FILE_MUST_EXIST) as fileDialog:
            if fileDialog.ShowModal() == wx.ID_CANCEL:
                return

            needthese_file = fileDialog.GetPath()

            with open(needthese_file, 'r') as file:
                lines = file.readlines()
                files_to_copy = []
                for line in lines:
                    parts = line.strip().split(') ', 1)
                    if len(parts) == 2:
                        files_to_copy.append((parts[0], parts[1]))

    def create_needthese_txt(self, event):
        global source_directory, destination_directory, files_to_copy

        source_directory = self.source_browse.GetValue()
        destination_directory = self.destination_browse.GetValue()

        if not source_directory or not destination_directory:
            wx.MessageBox("Please select both Source and Destination Directories.", "Folder Error", wx.OK | wx.ICON_WARNING)
            return

        source_files = self.get_files(source_directory)
        destination_files = self.get_files(destination_directory)

        files_to_copy = [file for file in source_files if file not in destination_files]

        if not files_to_copy:
            wx.MessageBox("No files need to be copied as they already exist in the destination folder.", "No Files to Copy", wx.OK | wx.ICON_INFORMATION)
            return

        with wx.FileDialog(self, "Save needthese.txt", wildcard="Text Files (*.txt)|*.txt",
                           style=wx.FD_SAVE | wx.FD_OVERWRITE_PROMPT) as fileDialog:
            if fileDialog.ShowModal() == wx.ID_CANCEL:
                return

            needthese_file = fileDialog.GetPath()

            with open(needthese_file, 'w') as file:
                for i, file_name in enumerate(files_to_copy, start=1):
                    file.write(f"{i}) {file_name}\n")

    def get_files(self, folder):
        files = []
        for root, _, filenames in os.walk(folder):
            for filename in filenames:
                if not filename.endswith(tuple(ignore_extensions)):
                    files.append(filename)
        return files

    def show_copy_summary(self, size_mismatch_files=None):
        global copied_files, skipped_files, not_found_files, move_files

        summary_text = f"Copied Files: {len(copied_files)} / {total_files_to_copy}\n"
        summary_text += "\n".join(copied_files) + "\n\n"
        summary_text += f"Skipped Files (already exist in the destination): {len(skipped_files)}\n"
        summary_text += "\n".join(skipped_files) + "\n\n"

        if not_found_files:
            summary_text += f"Files Not Found in Source:\n"
            summary_text += "\n".join(not_found_files)

        if move_files:
            summary_text += f"\n\nMoved Files: {len(copied_files)}"

        if size_mismatch_files:
            summary_text += f"\n\nFiles with size mismatch after copy:\n"
            summary_text += "\n".join(size_mismatch_files)
        elif not size_mismatch_files and total_files_to_copy > 0:
            summary_text += f"\n\nFile sizes are a match."

        wx.MessageBox(summary_text, "Copy Summary", wx.OK | wx.ICON_INFORMATION)

app = wx.App(False)
frame = FolderComparisonTool(None, wx.ID_ANY, "Folder Comparison Tool")
app.MainLoop()

If you wish to compile it to an exe you can use PowerShell. you just have to install pyinstaller

Then put the .py in this directory. Then in Powershell CD to the same directory

C:\Users\(username)\appdata\Local\Packages\PythonSoftwareFoundation.Python.(python version)_qbz5n2kfra8p0\LocalCache\local-packages\Python311\Scripts

Then run this command with whatever your .py is called.

.\pyinstaller --onefile -w 'copythese.py'

 

[binaryzero]

Robocopy "<source>" "<destination>" /L

Robocopy "<source>" "<destination>" /E /COPYALL

Robocopy "<source>" "<destination>" /mir

Robocopy "<source>" "<destination>" "<file to copy>"

Robocopy "<source>" "<destination>" "<*.extension>"

etc...

Wrap some PowerShell logic around it, you've got yourself a stew...

https://a.co/d/acdlSZf

[neufuse Veteran]

seems like a lot of work for something xxcopy and robocopy already do

[+Warwagon MVC]

On 27/09/2023 at 02:10, binaryzero said:

Robocopy "<source>" "<destination>" /L

Robocopy "<source>" "<destination>" /E /COPYALL

Robocopy "<source>" "<destination>" /mir

Robocopy "<source>" "<destination>" "<file to copy>"

Robocopy "<source>" "<destination>" "<*.extension>"

etc...

Wrap some PowerShell logic around it, you've got yourself a stew...

https://a.co/d/acdlSZf

 

Here's the thing, I don't know if because I'm dyslexic or not but I've tried in the past to learn all sorts of differing coding and scripting. It just never works out. So I'm extremely tickled that something like Chat GPT exists to do most of the coding for me. Python is good because, in a some of cases, I can at least tell what the code was trying to do and fix some stuff, but couldn't actually write any of it from scratch.

On 27/09/2023 at 07:41, neufuse said:

seems like a lot of work for something xxcopy and robocopy already do

Some may call it work, but I call it fun. I had a blast working with Chat GPT making this, and now that it's done, I can use it for my needs.