freealberta/map/instruct/LISTMONK_INTEGRATION_GUIDE.md

Listmonk Integration Guide

This guide covers the complete setup and usage of the Listmonk integration for the Map application.

Overview

The Map application integrates seamlessly with Listmonk, a self-hosted newsletter and mailing list manager, to provide advanced email marketing capabilities. This integration enables:

• Real-time subscriber synchronization
• Automated list segmentation
• Professional email campaigns
• Advanced analytics and reporting
• Bi-directional data sync

Prerequisites

1. Listmonk Instance: You need a running Listmonk instance (typically deployed via Docker Compose)
2. Admin Access: Listmonk admin credentials for API access
3. Network Connectivity: Map application must be able to reach Listmonk instance
4. Environment Configuration: Proper .env setup

Quick Setup

1. Configure Environment Variables

Add these variables to your .env file:

# Listmonk Integration
LISTMONK_URL=http://listmonk:9000
LISTMONK_USERNAME=admin
LISTMONK_PASSWORD=your-secure-password
LISTMONK_ENABLED=true
LISTMONK_SYNC_ON_STARTUP=true
LISTMONK_AUTO_CREATE_LISTS=true

Important Notes:

• Use http://listmonk:9000 for Docker Compose deployments (service name)
• Use http://localhost:9000 for local development
• Ensure the password matches your Listmonk admin password

2. Restart the Application

docker-compose restart map-viewer

3. Verify Integration

1. Check Admin Panel: Go to /admin.html and look for the "Listmonk Integration" section
2. Status Indicator: Green status means successful connection
3. View Logs: Check terminal for sync messages
4. Listmonk Dashboard: Verify lists were created in Listmonk

Automated List Management

The integration automatically creates and maintains these subscriber lists:

Core Lists

• All Locations - Every location with contact information
• All Users - Every registered Map application user

Support Level Lists

• Support Level 1 - Strongest supporters (Green markers)
• Support Level 2 - Moderate supporters (Yellow markers)
• Support Level 3 - Weak supporters (Orange markers)
• Support Level 4 - Opponents (Red markers)

Sign Status Lists

• Has Sign - Locations with campaign signs
• No Sign - Locations without signs

Combined Segmentation Lists

• Support 1 with Signs - Strong supporters who have signs
• Support 2 with Signs - Moderate supporters who have signs
• Support 3 with Signs - Weak supporters who have signs
• Support 4 with Signs - Opponents who have signs (for tracking)
• Support 1-3 - All positive supporters (Levels 1, 2, and 3)
• Sign Requests - Supporters without signs (potential sign placements)

Real-Time Sync Features

Automatic Synchronization

• New Location Added: Instantly added to appropriate lists
• Location Updated: Support level or sign status changes trigger list updates
• Location Deleted: Removed from all lists
• User Created: Added to "All Users" list
• User Deleted: Removed from all lists

Bi-Directional Sync

• Unsubscribes: When users unsubscribe in Listmonk, their records are updated in the Map database
• Email Updates: Email address changes sync both ways
• List Management: Changes in Listmonk are reflected in Map data

Admin Management Features

Status Dashboard

• Connection Status: Live indicator (🟢 Connected, 🔴 Error, ⚪ Disabled)
• Sync Statistics: Total subscribers, lists, and sync operations
• Last Sync Times: When each data type was last synchronized
• Error Notifications: Pop-up alerts for sync failures

Manual Operations

• Full Resync: Completely rebuild all lists from current data
• Selective Sync: Sync only locations or users
• Test Connection: Verify Listmonk connectivity
• Reinitialize: Delete and recreate all lists
• Progress Tracking: Visual progress bars for bulk operations

Bulk Sync Controls

// Available admin actions
POST /api/listmonk/sync/full        // Complete resync
POST /api/listmonk/sync/locations   // Sync locations only  
POST /api/listmonk/sync/users       // Sync users only
POST /api/listmonk/test-connection  // Test connectivity
POST /api/listmonk/reinitialize     // Recreate lists

Error Handling & Troubleshooting

Common Issues

Connection Failed

Symptoms: Red status indicator, "Connection failed" in logs Solutions:

1. Verify LISTMONK_URL is correct
2. Check Listmonk is running: docker-compose ps
3. Test network connectivity: curl http://listmonk:9000
4. Verify credentials in Listmonk admin panel

Sync Errors

Symptoms: Yellow status indicator, specific error messages in logs Solutions:

1. Check Listmonk API limits and performance
2. Verify subscriber data format (valid emails, etc.)
3. Review Listmonk logs: docker-compose logs listmonk
4. Try manual resync from admin panel

Missing Lists

Symptoms: Lists not appearing in Listmonk Solutions:

1. Ensure LISTMONK_AUTO_CREATE_LISTS=true
2. Restart application to trigger initialization
3. Use "Reinitialize" button in admin panel
4. Check Listmonk permissions and API access

Slow Sync Performance

Symptoms: Long delays during sync operations Solutions:

1. Monitor Listmonk server resources
2. Check network latency between services
3. Consider reducing sync frequency for large datasets
4. Review database performance (both NocoDB and Listmonk)

Debug Logging

Enable detailed logging by setting:

NODE_ENV=development

This provides verbose sync operation details in the terminal.

Email Campaign Workflow

1. Segment Selection

Use the automatically created lists in Listmonk to target specific audience segments:

• Fundraising: Target "Support Level 1" for donation requests
• Volunteer Recruitment: Target "Support 1-3" for volunteer opportunities
• Sign Placement: Target "Sign Requests" for new sign installations
• Opposition Research: Monitor "Support Level 4" engagement

2. Campaign Creation

Create campaigns in Listmonk using the segmented lists:

1. Go to Listmonk admin panel
2. Create new campaign
3. Select appropriate subscriber lists
4. Design email content
5. Schedule or send immediately

3. Analytics & Follow-up

• Track Opens: Monitor engagement by support level
• Click Tracking: Measure action rates across segments
• Unsubscribe Monitoring: Identify potential support level changes
• A/B Testing: Test messaging effectiveness across segments

Advanced Configuration

Custom List Creation

To create additional custom lists, modify the Listmonk service configuration:

// app/services/listmonk.js
const customLists = [
    {
        name: 'High Value Voters',
        description: 'Support Level 1 & 2 with voting history',
        // Add custom segmentation logic
    }
];

Sync Frequency Tuning

Adjust sync timing based on your needs:

# Sync immediately on data changes (default: true)
LISTMONK_REALTIME_SYNC=true

# Startup sync (default: true) 
LISTMONK_SYNC_ON_STARTUP=true

# Batch size for large syncs (default: 100)
LISTMONK_BATCH_SIZE=50

Webhook Integration

For advanced use cases, consider setting up Listmonk webhooks to trigger Map application updates:

# Enable webhook endpoint
LISTMONK_WEBHOOK_ENABLED=true
LISTMONK_WEBHOOK_SECRET=your-webhook-secret

Security Considerations

Credentials Management

• Store securely: Use environment variables, never commit passwords
• Rotate regularly: Change Listmonk passwords periodically
• Limit access: Create dedicated API user with minimal required permissions
• Network security: Use HTTPS in production, restrict network access

Data Privacy

• GDPR Compliance: Ensure proper consent and unsubscribe handling
• Data Minimization: Only sync necessary contact information
• Retention Policies: Configure appropriate data retention in Listmonk
• Audit Logging: Monitor access and changes to subscriber data

Integration Monitoring

Health Checks

The system provides several monitoring endpoints:

GET /api/listmonk/status        // Connection and sync status
GET /api/listmonk/lists         // List information and counts
GET /health                     // Overall application health

Performance Metrics

Monitor these key metrics:

• Sync Duration: Time to complete full synchronization
• Error Rate: Percentage of failed sync operations
• List Growth: Subscriber count changes over time
• API Response Time: Listmonk API performance

Alerting

Set up monitoring for:

• Connection failures (implement external monitoring)
• Sync errors (check application logs)
• List count discrepancies (compare Map vs Listmonk counts)
• Performance degradation (sync time increases)

Backup & Recovery

Data Backup

• Listmonk Data: Regular database backups of Listmonk
• Map Data: NocoDB backup includes all source data
• Configuration: Version control .env files (without secrets)

Disaster Recovery

In case of data loss:

1. Restore Listmonk: From database backup
2. Reinitialize: Use admin panel "Reinitialize" function
3. Full Resync: Rebuild all lists from Map data
4. Verify: Check list counts and subscriber data

Best Practices

Campaign Management

• Consistent Messaging: Maintain brand voice across all campaigns
• Segmentation Strategy: Use meaningful, actionable segments
• Testing: A/B test subject lines and content
• Compliance: Follow CAN-SPAM and GDPR requirements
• Analytics: Track and analyze campaign performance

Data Quality

• Email Validation: Verify email addresses before adding to lists
• Duplicate Prevention: System handles duplicates automatically
• Regular Cleanup: Remove bounced and inactive subscribers
• Contact Updates: Keep contact information current

Performance Optimization

• Batch Operations: Use bulk sync for large data changes
• Off-Peak Sync: Schedule heavy operations during low-usage periods
• Resource Monitoring: Watch server resources during sync operations
• Network Optimization: Ensure reliable connectivity between services

Support & Documentation

Additional Resources

• Listmonk Documentation: https://listmonk.app/docs/
• Map Application Docs: See main README.md
• API Reference: Available at /api endpoints
• Community Support: GitHub issues and discussions

Getting Help

If you encounter issues:

1. Check Status: Admin panel shows current integration status
2. Review Logs: Terminal output provides detailed error information
3. Test Connection: Use admin panel connection test
4. Documentation: Refer to this guide and Listmonk docs
5. Community: Ask questions in project GitHub issues

This integration significantly enhances the Map application's email marketing capabilities while maintaining data consistency and providing powerful segmentation options for targeted campaigns.