Prompt 1

- Description: This project revolves around creating a set of python scripts which we will use to parse CSV data. The CSV data represents Google Business Profile data. It was scraped from Google Maps. Our purpose is to clean this data up and prepare it for use with a toolkit of custom made python scripts. - Project Root: "/home/ld/mgk-scrapes" - Current Dataset: "/current-data/" - Data Directory: "/.data/" - Parent Data: "/"(All directories that do not begin with a period ".") - Stage Directories: "/stage-#" (Where # is a number.) - Stage 1: "/stage-1/" - Stage 2: "/stage-2/" - Stage 3: "/stage-3/" - Stage 4: "/stage-4/" - Stage 5: "/stage-5/" - Stage 6: "/stage-6/" - Binaries: "/bin/" - Stage 1 Binaries: "/stage-1/" - Stage 2 Binaries: "/stage-2/" - Stage 3 Binaries: "/stage-3/" - Stage 4 Binaries: "/stage-4/" - Stage 5 Binaries: "/stage-5/" - Stage 6 Binaries: "/stage-6/" - Rule 1: All scripts need to be able to be ran from any directory. - Rule 2: All scripts need output what they are doing, as they are doing it. - Rule 3: All tasks should use halo to report successes and failures. - Rule 4: The python module "tqdm" can be used to report progress when appropriate. - Rule 5: Scripts must be heavily commented, describing the purpose of the script and what each code block inside the code is for. - Rule 6: Every script should begin with a hashbang "#!/usr/bin/env python". - Rule 7: Every script should have a name, and the name of the script should be immediately beneath the hashbang in a comment field prefixed by a string that reads "Script Name: ". - Rule 8: Scripts should generally have robust error checking.

I need help creating a new script. See above for metadata and information regarding our project setup, configuration, purpose, and general rules to follow.

This script should be called "delete-empty-counties".

This script should iterate through the sub directories in <Stage 1> looking for empty sub-directories and then delete them.

<Stage 1> consists of and beneath the .

Sometimes these are empty of CSV files. When that is the case, we want to delete those directories.

The script should report what it is doing.

It should report "Empty counties found:" and then on the next line, provide a comma separated list of all we found that have no CSV files which we are going to delete.

It should then report that it is deleting the directories, and then delete them.

At the end, report a final tally of all directories deleted.

This script will reside in the directory, and needs to be able to be ran against different stages.

So it will need a command line option provided to it, such as these examples:

delete-empty-counties.py stage-1 delete-empty-counties.py stage-2

Prompt 2

I got this error:

Error: The directory stage-1 does not exist.

Please ensure that the have been referenced to setup the correct variables at the start of the script so that the script knows where to find the .

Prompt 3

- [Description]: This project revolves around creating a set of python scripts which we will use to parse CSV data. The CSV data represents Google Business Profile data. It was scraped from Google Maps. Our purpose is to clean this data up and prepare it for use with a toolkit of custom made python scripts. - [Project Root]: "/home/ld/mgk-scrapes" - [Current Dataset]: "[Project Root]/current-data/" - [Data Directory]: "[Current Dataset]/.data/" - [Parent Data]: "[Current Dataset]/"(All directories that do not begin with a period ".") - [Stage Directories]: "[Data Directory]/stage-#" (Where # is a number.) - [State Directories]: The state directories are the directories directly under one of the [Stage Directories]. These directories store data associated with a specific state. - [County Directories]: The coiunty directories are the directories directly under one of the [State Directories]. These directories store data associated with a specific county inside of the state the county directory is under. - [Stage 1]: "[Data Directory]/stage-1/" - [Stage 2]: "[Data Directory]/stage-2/" - [Stage 3]: "[Data Directory]/stage-3/" - [Stage 4]: "[Data Directory]/stage-4/" - [Stage 5]: "[Data Directory]/stage-5/" - [Stage 6]: "[Data Directory]/stage-6/" - [Binaries]: "[Project Root]/bin/" - [Stage 1 Binaries]: "[Binaries]/stage-1/" - [Stage 2 Binaries]: "[Binaries]/stage-2/" - [Stage 3 Binaries]: "[Binaries]/stage-3/" - [Stage 4 Binaries]: "[Binaries]/stage-4/" - [Stage 5 Binaries]: "[Binaries]/stage-5/" - [Stage 6 Binaries]: "[Binaries]/stage-6/" - [Bad Matching Services]: "[Data Directory]/bad-matching-services.txt" - [GBP Business Categories]: "[Data Directory]/gbp-business-categories.txt" - [GBP Matching Services]: "[Data Directory]/gbp-matching-services.txt" - [Unknown Blacklist]: "[Data Directory]/unknown-blacklist.txt" - [Rule 1]: All scripts need to be able to be ran from any directory. - [Rule 2]: All scripts need output what they are doing, as they are doing it. - [Rule 3]: All tasks should use halo to report successes and failures. - [Rule 4]: The python module "tqdm" can be used to report progress when appropriate. - [Rule 5]: Scripts must be heavily commented, describing the purpose of the script and what each code block inside the code is for. - [Rule 6]: Every script should begin with a hashbang "#!/usr/bin/env python". - [Rule 7]: Every script should have a name, and the name of the script should be immediately beneath the hashbang in a comment field prefixed by a string that reads "Script Name: ". - [Rule 8]: Scripts should generally have robust error checking.

Please be sure to reference project details above when factoring code, or answering questions.

I wish to modify the "delete-empty-columns" script:

This script should use pyfiglet to output the name of the script as it runs.

Prompt 4

Can the pyfiglet portion of this script be made to strip hyphens from the script name before using pyfiglet to display the name of the script?

Prompt 5

Ok I would like the hypens actually replaced with spaces, not stripped or removed.

Prompt 6

Additionally, after the hyphens have been replaced with spaces, I would like the words that are left to be capitalized before printing the pyfiglet.

Prompt 7

I wish to modify the 'delete-empty-counties' script:

#!/usr/bin/env python # Script Name: delete-empty-counties

import os import sys import shutil import halo from pyfiglet import Figlet

Define the paths based on the project details

PROJECT_ROOT = "/home/ld/mgk-scrapes" CURRENT_DATASET = os.path.join(PROJECT_ROOT, "current-data") DATA_DIRECTORY = os.path.join(CURRENT_DATASET, ".data")

def get_stage_directory(stage): """Get the path for the specified stage directory.""" return os.path.join(DATA_DIRECTORY, stage)

def find_empty_counties(stage_directory): """Find all empty county directories in the given stage directory.""" empty_counties = [] for state_dir in os.listdir(stage_directory): state_path = os.path.join(stage_directory, state_dir) if os.path.isdir(state_path): for county_dir in os.listdir(state_path): county_path = os.path.join(state_path, county_dir) if os.path.isdir(county_path): # Check if the county directory contains any CSV files csv_files = [f for f in os.listdir(county_path) if f.endswith('.csv')] if not csv_files: empty_counties.append(county_path) return empty_counties

def delete_directories(directories): """Delete the given directories.""" for directory in directories: shutil.rmtree(directory)

def main(): figlet = Figlet(font='slant') script_name = "delete-empty-counties".replace("-", " ").title() print(figlet.renderText(script_name))

if len(sys.argv) != 2:
    print("Usage: delete-empty-counties.py <stage-directory>")
    sys.exit(1)

stage = sys.argv[1]
stage_directory = get_stage_directory(stage)

if not os.path.exists(stage_directory):
    print(f"Error: The directory {stage_directory} does not exist.")
    sys.exit(1)

print(f"Scanning for empty counties in {stage_directory}...")
spinner = halo.Halo(text='Scanning', spinner='dots')
spinner.start()
empty_counties = find_empty_counties(stage_directory)
spinner.succeed("Scan complete.")

if empty_counties:
    print("Empty counties found:")
    print(", ".join(empty_counties))
    
    print("Deleting empty counties...")
    spinner.start()
    delete_directories(empty_counties)
    spinner.succeed("Deletion complete.")
    
    print(f"Total directories deleted: {len(empty_counties)}")
else:
    print("No empty counties found.")

if name == "main": main() </delete-empty-counties script>

Change 1: Right now it it deletes empty county directories, but I would like it to also delete empty state directories in the stage folder being processed.

15 KiB Raw Blame History