# Plain Keyword List Import Documentation

## Overview
These scripts handle the import of plain keyword lists, typically from existing ad groups and campaigns. This process differs from GKWP and SEMRush imports as it:
- Doesn't include metric data
- Supports direct ad group assignment
- Can create new ad groups during import
- Doesn't use seed keywords
- Focuses on organization rather than data enrichment

## File Format Requirements
- One keyword per line text file
- Optional CSV format with ad group assignments
- Supported formats:
  ```
  # Simple text file
  keyword one
  keyword two
  keyword three

  # CSV with ad group assignments
  keyword,ad_group_name,campaign_name
  keyword one,products,main campaign
  keyword two,services,main campaign
  ```

## Script Locations
```
/bin/import-scripts/plain-list/
├── import-keywords.sh            # Basic keyword list import
├── import-with-adgroups.sh      # Import with ad group assignments
├── create-adgroup-import.sh     # Import + create new ad group
└── sql/
    ├── import_keywords.sql      # Basic import processing
    ├── assign_adgroups.sql      # Ad group assignment
    └── create_adgroup.sql       # New ad group creation
```

## Import Process Flow
```mermaid
graph TD
    A[Read Input File] --> B{Format Check}
    B -->|Plain List| C[Basic Import]
    B -->|CSV with AdGroups| D[Parse Assignments]
    D --> E{AdGroup Exists?}
    E -->|Yes| F[Assign Keywords]
    E -->|No| G[Create AdGroup]
    G --> F
    C --> H[Update Import History]
    F --> H
```

## Basic Import Script

### Location
`/bin/import-scripts/plain-list/import-keywords.sh`

### Implementation
```bash
#!/bin/bash

# Load configuration
source ../../../.database

# Set error handling
set -e

# Setup logging
LOG_DIR="../../../logs"
LOG_FILE="${LOG_DIR}/$(date +%Y-%m-%d)-plain-import.log"

mkdir -p "$LOG_DIR"

# Function to log messages
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

# Check arguments
if [ $# -lt 1 ]; then
    echo "Usage: $0 <keyword_file> [ad_group_id]"
    exit 1
fi

INPUT_FILE=$1
AD_GROUP_ID=$2

# Create import history record
IMPORT_ID=$(psql -h "${DB_HOST}" \
                 -p "${DB_PORT}" \
                 -U "${DB_USER}" \
                 -d "${DB_NAME}" \
                 -tAc "INSERT INTO Import_History 
                       (source_type, filename, assigned_ad_group_id, import_status) 
                       VALUES ('PLAIN_LIST', '$(basename "$INPUT_FILE")', 
                               ${AD_GROUP_ID:=NULL}, 'IN_PROGRESS') 
                       RETURNING import_id;")

# Process file
log_message "Processing keyword list: $(basename "$INPUT_FILE")"

# Import keywords
psql -h "${DB_HOST}" \
     -p "${DB_PORT}" \
     -U "${DB_USER}" \
     -d "${DB_NAME}" \
     -v import_id="$IMPORT_ID" \
     -v ad_group_id="$AD_GROUP_ID" \
     -f "sql/import_keywords.sql"

# Get results
RESULTS=$(psql -h "${DB_HOST}" \
               -p "${DB_PORT}" \
               -U "${DB_USER}" \
               -d "${DB_NAME}" \
               -tAc "SELECT success_count, error_count 
                     FROM Import_History 
                     WHERE import_id = ${IMPORT_ID};")

log_message "Import completed: ${RESULTS}"
```

### SQL Import Script
```sql
-- sql/import_keywords.sql

-- Create temporary table for import
CREATE TEMP TABLE keyword_import (
    keyword TEXT
);

-- Import data
COPY keyword_import FROM STDIN WITH (FORMAT text);

-- Process keywords
WITH inserted_keywords AS (
    INSERT INTO Keywords (keyword)
    SELECT DISTINCT LOWER(keyword)
    FROM keyword_import
    ON CONFLICT (LOWER(keyword)) DO NOTHING
    RETURNING keyword_id
),
all_keywords AS (
    SELECT k.keyword_id
    FROM Keywords k
    JOIN keyword_import ki ON LOWER(ki.keyword) = k.keyword
)
INSERT INTO Keyword_Ad_Group_Assignments (
    keyword_id,
    ad_group_id,
    status
)
SELECT 
    keyword_id,
    :ad_group_id,
    'active'
FROM all_keywords
WHERE :ad_group_id IS NOT NULL;

-- Update import history
UPDATE Import_History
SET 
    import_status = 'SUCCESS',
    success_count = (SELECT COUNT(*) FROM keyword_import),
    error_count = 0
WHERE import_id = :import_id;

-- Cleanup
DROP TABLE keyword_import;
```

## Ad Group Import Script

### Location
`/bin/import-scripts/plain-list/import-with-adgroups.sh`

### Implementation
```bash
#!/bin/bash

# Enhanced version supporting ad group assignments
# Implementation details for ad group import...
```

## Create and Import Script

### Location
`/bin/import-scripts/plain-list/create-adgroup-import.sh`

### Usage
```bash
# Create new ad group and import keywords
./create-adgroup-import.sh keyword_file.txt "New Ad Group" campaign_id landing_page_id

# Import to existing ad group
./import-with-adgroups.sh keyword_file.csv
```

## Testing
Located in `/tests/import-scripts/plain-list/`

Example test:
```bash
#!/bin/bash

source ../../../utils/test-framework.sh

test_plain_import() {
    # Create test campaign and ad group
    campaign_id=$(psql -tAc "INSERT INTO Campaigns (campaign_name) VALUES ('Test Campaign') RETURNING campaign_id;")
    landing_page_id=$(psql -tAc "INSERT INTO Landing_Pages (url, page_name) VALUES ('http://test.com', 'Test Page') RETURNING landing_page_id;")
    ad_group_id=$(psql -tAc "INSERT INTO Ad_Groups (campaign_id, landing_page_id, ad_group_name) VALUES ($campaign_id, $landing_page_id, 'Test Group') RETURNING ad_group_id;")
    
    # Create test file
    cat > "test-data/keywords.txt" << EOL
test keyword one
test keyword two
test keyword three
EOL

    # Run import
    ../bin/import-scripts/plain-list/import-keywords.sh "test-data/keywords.txt" "$ad_group_id"
    
    # Verify imports
    result=$(psql -tAc "SELECT COUNT(*) FROM Keywords WHERE keyword LIKE 'test keyword%';" "${DB_NAME}")
    assert_equals "$result" "3" "Should import all keywords"
    
    # Verify assignments
    assignments=$(psql -tAc "SELECT COUNT(*) FROM Keyword_Ad_Group_Assignments WHERE ad_group_id = $ad_group_id;" "${DB_NAME}")
    assert_equals "$assignments" "3" "Should assign all keywords to ad group"
    
    # Clean up
    rm "test-data/keywords.txt"
}

test_adgroup_creation() {
    # Test creating new ad group during import
    # Implementation of ad group creation test...
}

run_test_suite
```

## Error Handling

1. File Validation
   - Line format checking
   - Character encoding
   - Duplicate detection

2. Ad Group Validation
   - Existence checking
   - Permission verification
   - Campaign validation

3. Assignment Validation
   - Duplicate assignments
   - Status tracking
   - Constraint checking

## Best Practices

1. Data Preparation
   - Trim whitespace
   - Convert to lowercase
   - Remove duplicates
   - Validate keywords

2. Performance
   - Batch processing
   - Transaction management
   - Index utilization

3. Organization
   - Clear naming conventions
   - Consistent status values
   - Proper documentation

4. Verification
   - Import confirmation
   - Assignment verification
   - Status checking

## Integration with LibreOffice Base

### Views for Import Verification
```sql
CREATE VIEW v_recent_imports AS
SELECT 
    ih.import_date,
    ih.filename,
    ih.success_count,
    ag.ad_group_name,
    c.campaign_name
FROM Import_History ih
LEFT JOIN Ad_Groups ag ON ih.assigned_ad_group_id = ag.ad_group_id
LEFT JOIN Campaigns c ON ag.campaign_id = c.campaign_id
WHERE ih.source_type = 'PLAIN_LIST'
ORDER BY ih.import_date DESC;
```