hirejared-kw-db/docs/project-docs/05-operations/troubleshooting.md

Troubleshooting Guide

Common Issues and Solutions

Database Connection Issues

Unable to Connect to Database

Error: could not connect to server: Connection refused

Possible Causes:

1. PostgreSQL service not running
2. Incorrect connection details
3. Network/firewall issues

Solutions:

1. Check PostgreSQL service:

   # Check service status
   sudo systemctl status postgresql
   
   # Start service if stopped
   sudo systemctl start postgresql
   

2. Verify connection details in .database:

   # Test connection manually
   psql -h "${DB_HOST}" -p "${DB_PORT}" -U "${DB_USER}" -d "${DB_NAME}"
   

3. Check firewall settings:

   # Check if port is open
   sudo netstat -tulpn | grep 5432
   

Authentication Failed

Error: FATAL: password authentication failed for user

Solutions:

1. Verify credentials in .database
2. Check PostgreSQL authentication settings:

   # View pg_hba.conf location
   psql -U postgres -c 'SHOW hba_file;'
   
   # Check user permissions
   psql -U postgres -c '\du'
   

Import Process Issues

GKWP Import Failures

File Format Errors:

Error: Invalid filename format. Expected: GKWP - {KEYWORD}.csv

Solutions:

1. Check file naming convention
2. Verify file content format:

   # View file headers
   head -n 1 "GKWP - keyword.csv"
   
   # Check file encoding
   file -i "GKWP - keyword.csv"
   

Data Processing Errors:

Error: Invalid data format in row X

Solutions:

1. Examine problematic rows:

   # View specific line
   sed -n 'Xp' "GKWP - keyword.csv"
   

2. Check for special characters:

   # Look for non-ASCII characters
   grep -P '[^\x00-\x7F]' "GKWP - keyword.csv"
   

SEMRush Import Issues

Duplicate Keywords:

Error: duplicate key value violates unique constraint

Solutions:

1. Check existing keywords:

   SELECT keyword 
   FROM Keywords 
   WHERE LOWER(keyword) = LOWER('problematic_keyword');
   

2. Use conflict resolution:

   INSERT ... ON CONFLICT (LOWER(keyword)) DO UPDATE ...
   

LibreOffice Base Connection Issues

Unable to Connect

Symptoms:

• Connection error dialog
• Database not showing in available connections

Solutions:

1. Verify PostgreSQL ODBC/JDBC settings
2. Check LibreOffice Base configuration:
   • Database URL format
   • Driver selection
   • Connection parameters

Slow Performance

Solutions:

1. Check indexes:

   -- View missing indexes
   SELECT schemaname, tablename, reason, round(percent,2)
   FROM pg_stat_user_tables
   WHERE n_live_tup > 100000
   ORDER BY n_live_tup DESC;
   

2. Optimize views:

   -- Analyze view performance
   EXPLAIN ANALYZE SELECT * FROM v_active_keywords_per_ad_group;
   

Script Execution Issues

Permission Denied

bash: ./bin/db-admin/create-db.sh: Permission denied

Solutions:

1. Check file permissions:

   # View permissions
   ls -l bin/db-admin/create-db.sh
   
   # Set correct permissions
   chmod +x bin/db-admin/create-db.sh
   

2. Verify script ownership:

   # Change ownership if needed
   chown proper_user:proper_group bin/db-admin/create-db.sh
   

Path Issues

Error: Could not source .database file

Solutions:

1. Check script execution directory
2. Use absolute paths:

   # Get script directory
   SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
   source "${SCRIPT_DIR}/../../.database"
   

Data Integrity Issues

Orphaned Records

Symptoms:

• Missing relationships
• Incomplete data

Solutions:

1. Check for orphaned records:

   -- Find keywords without ad groups
   SELECT k.keyword_id, k.keyword
   FROM Keywords k
   LEFT JOIN Keyword_Ad_Group_Assignments kaga 
   ON k.keyword_id = kaga.keyword_id
   WHERE kaga.ad_group_id IS NULL;
   

2. Clean up orphaned data:

   -- Remove orphaned assignments
   DELETE FROM Keyword_Ad_Group_Assignments
   WHERE ad_group_id NOT IN (SELECT ad_group_id FROM Ad_Groups);
   

Logging and Monitoring

Missing Logs

Solutions:

1. Check log directory permissions
2. Verify log configuration:

   # Check log directory
   ls -la logs/
   
   # Create log directory if missing
   mkdir -p logs/
   chmod 750 logs/
   

Error Investigation

# Search for errors in logs
grep -r "ERROR" logs/

# View recent errors
tail -f logs/$(date +%Y-%m-%d)-*.log

Preventive Measures

Regular Maintenance

1. Database cleanup:

   VACUUM ANALYZE;
   

2. Log rotation:

   # Rotate logs older than 30 days
   find logs/ -name "*.log" -mtime +30 -delete
   

3. Backup verification:

   # Test backup integrity
   ./bin/db-admin/verify-backup.sh backups/latest.sql.gz
   

Monitoring

1. Database size monitoring:

   SELECT pg_size_pretty(pg_database_size('keyword_management'));
   

2. Table growth monitoring:

   SELECT relname, n_live_tup
   FROM pg_stat_user_tables
   ORDER BY n_live_tup DESC;
   

Emergency Procedures

Database Recovery

1. Stop all active processes
2. Restore from latest backup:

   ./bin/db-admin/restore-db.sh backups/latest.sql.gz
   

Data Corruption

1. Switch to read-only mode:

   ALTER DATABASE keyword_management SET default_transaction_read_only = on;
   

2. Investigate issues:

   -- Check table integrity
   SELECT * FROM pg_stat_database WHERE datname = 'keyword_management';
   

Support Resources

1. Log locations:

   • Script logs: /logs
   • PostgreSQL logs: (system-dependent)
   • LibreOffice Base logs

2. Configuration files:

   • .database
   • PostgreSQL configuration
   • Script configurations

Would you like me to expand on any particular troubleshooting aspect or add more examples?