hirejared-kw-db/docs/project-docs/03-scripts/import-scripts/plain-list-import.md

Plain Keyword List Import Documentation

Overview

These scripts handle the import of plain keyword lists, typically from existing ad groups and campaigns. This process differs from GKWP and SEMRush imports as it:

• Doesn't include metric data
• Supports direct ad group assignment
• Can create new ad groups during import
• Doesn't use seed keywords
• Focuses on organization rather than data enrichment

File Format Requirements

• One keyword per line text file
• Optional CSV format with ad group assignments
• Supported formats:

  # Simple text file
  keyword one
  keyword two
  keyword three
  
  # CSV with ad group assignments
  keyword,ad_group_name,campaign_name
  keyword one,products,main campaign
  keyword two,services,main campaign
  

Script Locations

/bin/import-scripts/plain-list/
├── import-keywords.sh            # Basic keyword list import
├── import-with-adgroups.sh      # Import with ad group assignments
├── create-adgroup-import.sh     # Import + create new ad group
└── sql/
    ├── import_keywords.sql      # Basic import processing
    ├── assign_adgroups.sql      # Ad group assignment
    └── create_adgroup.sql       # New ad group creation

Import Process Flow

graph TD
    A[Read Input File] --> B{Format Check}
    B -->|Plain List| C[Basic Import]
    B -->|CSV with AdGroups| D[Parse Assignments]
    D --> E{AdGroup Exists?}
    E -->|Yes| F[Assign Keywords]
    E -->|No| G[Create AdGroup]
    G --> F
    C --> H[Update Import History]
    F --> H

Basic Import Script

Location

/bin/import-scripts/plain-list/import-keywords.sh

Implementation

#!/bin/bash

# Load configuration
source ../../../.database

# Set error handling
set -e

# Setup logging
LOG_DIR="../../../logs"
LOG_FILE="${LOG_DIR}/$(date +%Y-%m-%d)-plain-import.log"

mkdir -p "$LOG_DIR"

# Function to log messages
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check arguments
if [ $# -lt 1 ]; then
    echo "Usage: $0 <keyword_file> [ad_group_id]"
    exit 1
fi

INPUT_FILE=$1
AD_GROUP_ID=$2

# Create import history record
IMPORT_ID=$(psql -h "${DB_HOST}" \
                 -p "${DB_PORT}" \
                 -U "${DB_USER}" \
                 -d "${DB_NAME}" \
                 -tAc "INSERT INTO Import_History 
                       (source_type, filename, assigned_ad_group_id, import_status) 
                       VALUES ('PLAIN_LIST', '$(basename "$INPUT_FILE")', 
                               ${AD_GROUP_ID:=NULL}, 'IN_PROGRESS') 
                       RETURNING import_id;")

# Process file
log_message "Processing keyword list: $(basename "$INPUT_FILE")"

# Import keywords
psql -h "${DB_HOST}" \
     -p "${DB_PORT}" \
     -U "${DB_USER}" \
     -d "${DB_NAME}" \
     -v import_id="$IMPORT_ID" \
     -v ad_group_id="$AD_GROUP_ID" \
     -f "sql/import_keywords.sql"

# Get results
RESULTS=$(psql -h "${DB_HOST}" \
               -p "${DB_PORT}" \
               -U "${DB_USER}" \
               -d "${DB_NAME}" \
               -tAc "SELECT success_count, error_count 
                     FROM Import_History 
                     WHERE import_id = ${IMPORT_ID};")

log_message "Import completed: ${RESULTS}"

SQL Import Script

-- sql/import_keywords.sql

-- Create temporary table for import
CREATE TEMP TABLE keyword_import (
    keyword TEXT
);

-- Import data
COPY keyword_import FROM STDIN WITH (FORMAT text);

-- Process keywords
WITH inserted_keywords AS (
    INSERT INTO Keywords (keyword)
    SELECT DISTINCT LOWER(keyword)
    FROM keyword_import
    ON CONFLICT (LOWER(keyword)) DO NOTHING
    RETURNING keyword_id
),
all_keywords AS (
    SELECT k.keyword_id
    FROM Keywords k
    JOIN keyword_import ki ON LOWER(ki.keyword) = k.keyword
)
INSERT INTO Keyword_Ad_Group_Assignments (
    keyword_id,
    ad_group_id,
    status
)
SELECT 
    keyword_id,
    :ad_group_id,
    'active'
FROM all_keywords
WHERE :ad_group_id IS NOT NULL;

-- Update import history
UPDATE Import_History
SET 
    import_status = 'SUCCESS',
    success_count = (SELECT COUNT(*) FROM keyword_import),
    error_count = 0
WHERE import_id = :import_id;

-- Cleanup
DROP TABLE keyword_import;

Ad Group Import Script

Location

/bin/import-scripts/plain-list/import-with-adgroups.sh

Implementation

#!/bin/bash

# Enhanced version supporting ad group assignments
# Implementation details for ad group import...

Create and Import Script

Location

/bin/import-scripts/plain-list/create-adgroup-import.sh

Usage

# Create new ad group and import keywords
./create-adgroup-import.sh keyword_file.txt "New Ad Group" campaign_id landing_page_id

# Import to existing ad group
./import-with-adgroups.sh keyword_file.csv

Testing

Located in /tests/import-scripts/plain-list/

Example test:

#!/bin/bash

source ../../../utils/test-framework.sh

test_plain_import() {
    # Create test campaign and ad group
    campaign_id=$(psql -tAc "INSERT INTO Campaigns (campaign_name) VALUES ('Test Campaign') RETURNING campaign_id;")
    landing_page_id=$(psql -tAc "INSERT INTO Landing_Pages (url, page_name) VALUES ('http://test.com', 'Test Page') RETURNING landing_page_id;")
    ad_group_id=$(psql -tAc "INSERT INTO Ad_Groups (campaign_id, landing_page_id, ad_group_name) VALUES ($campaign_id, $landing_page_id, 'Test Group') RETURNING ad_group_id;")
    
    # Create test file
    cat > "test-data/keywords.txt" << EOL
test keyword one
test keyword two
test keyword three
EOL

    # Run import
    ../bin/import-scripts/plain-list/import-keywords.sh "test-data/keywords.txt" "$ad_group_id"
    
    # Verify imports
    result=$(psql -tAc "SELECT COUNT(*) FROM Keywords WHERE keyword LIKE 'test keyword%';" "${DB_NAME}")
    assert_equals "$result" "3" "Should import all keywords"
    
    # Verify assignments
    assignments=$(psql -tAc "SELECT COUNT(*) FROM Keyword_Ad_Group_Assignments WHERE ad_group_id = $ad_group_id;" "${DB_NAME}")
    assert_equals "$assignments" "3" "Should assign all keywords to ad group"
    
    # Clean up
    rm "test-data/keywords.txt"
}

test_adgroup_creation() {
    # Test creating new ad group during import
    # Implementation of ad group creation test...
}

run_test_suite

Error Handling

1. File Validation

   • Line format checking
   • Character encoding
   • Duplicate detection

2. Ad Group Validation

   • Existence checking
   • Permission verification
   • Campaign validation

3. Assignment Validation

   • Duplicate assignments
   • Status tracking
   • Constraint checking

Best Practices

1. Data Preparation

   • Trim whitespace
   • Convert to lowercase
   • Remove duplicates
   • Validate keywords

2. Performance

   • Batch processing
   • Transaction management
   • Index utilization

3. Organization

   • Clear naming conventions
   • Consistent status values
   • Proper documentation

4. Verification

   • Import confirmation
   • Assignment verification
   • Status checking

Integration with LibreOffice Base

Views for Import Verification

CREATE VIEW v_recent_imports AS
SELECT 
    ih.import_date,
    ih.filename,
    ih.success_count,
    ag.ad_group_name,
    c.campaign_name
FROM Import_History ih
LEFT JOIN Ad_Groups ag ON ih.assigned_ad_group_id = ag.ad_group_id
LEFT JOIN Campaigns c ON ag.campaign_id = c.campaign_id
WHERE ih.source_type = 'PLAIN_LIST'
ORDER BY ih.import_date DESC;