freealberta/map/README.md

609 lines
22 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# NocoDB Map Viewer
A containerized web application that visualizes geographic data from NocoDB on an interactive map using Leaflet.js.
## Features
- 🗺️ Interactive map visualization with OpenStreetMap
- 🔍 **Unified search system with docs and address search** (Ctrl+K to activate)
- 📍 Real-time geolocation support
- Add new locations directly from the map
- 🔄 Auto-refresh every 30 seconds
- 📱 Responsive design for mobile devices
- 🔒 Secure API proxy to protect credentials
- 👤 User authentication with login system
- ⚙️ Admin panel for system configuration
- 🎯 Configurable map start location
- 📋 Walk Sheet generator for door-to-door canvassing
- 🔗 QR code integration for digital resources
- 📅 Volunteer shift management system with calendar and grid views
- ✋ User shift signup and cancellation with color-coded calendar
- 👥 Admin shift creation and management
- 👨‍💼 User management panel for admin users (create, delete users)
- 🔐 Role-based access control (Admin vs User permissions)
- 📧 Email notifications and password recovery via SMTP
- 📊 CSV data import with batch geocoding and visual progress tracking
- 🐳 Docker containerization for easy deployment
- 🆓 100% open source (no proprietary dependencies)
## Quick Start
### Prerequisites
- Docker and Docker Compose
- NocoDB instance with API access
- NocoDB API token
### Installation
1. **Get NocoDB API Token**
1. Login to your NocoDB instance
2. Click user icon → **Account Settings****API Tokens**
3. Create new token with read/write permissions
4. Copy the token for the next step
2. **Configure Environment**
Edit the `.env` file with your NocoDB API and API Url:
```env
# NocoDB API Configuration
NOCODB_API_URL=https://db.cmlite.org/api/v1
NOCODB_API_TOKEN=your-api-token-here
# These will be populated after running build-nocodb.sh
NOCODB_VIEW_URL=
NOCODB_LOGIN_SHEET=
NOCODB_SETTINGS_SHEET=
NOCODB_SHIFTS_SHEET=
NOCODB_SHIFT_SIGNUPS_SHEET=
# Domain Configuration
DOMAIN=cmlite.org
# MkDocs Integration
MKDOCS_URL=https://cmlite.org
MKDOCS_SEARCH_URL=https://cmlite.org
MKDOCS_SITE_SERVER_PORT=4002
# Server Configuration
PORT=3000
NODE_ENV=production
# Session Secret (Generate with: openssl rand -hex 32)
SESSION_SECRET=your-secure-random-string
# Map Defaults (Edmonton, Alberta, Canada)
DEFAULT_LAT=53.5461
DEFAULT_LNG=-113.4938
DEFAULT_ZOOM=11
# Optional: Map Boundaries (prevents users from adding points outside area)
# BOUND_NORTH=53.7
# BOUND_SOUTH=53.4
# BOUND_EAST=-113.3
# BOUND_WEST=-113.7
# Cloudflare Settings
TRUST_PROXY=true
COOKIE_DOMAIN=.cmlite.org
# Allowed Origins
ALLOWED_ORIGINS=https://map.cmlite.org,http://localhost:3000
# Email Configuration (Optional - for password recovery)
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_SECURE=false
SMTP_USER=your-email@gmail.com
SMTP_PASS=your-app-password
EMAIL_FROM_NAME=CMlite Support
EMAIL_FROM_ADDRESS=noreply@cmlite.org
APP_NAME=CMlite Map
```
3. **Auto-Create Database Structure**
Run the build script to create required tables:
```bash
chmod +x build-nocodb.sh
./build-nocodb.sh
```
This creates five tables:
- **Locations** - Main map data with geo-location, contact info, support levels
- **Login** - User authentication (email, name, admin flag)
- **Settings** - Admin configuration and QR codes
- **Shifts** - Shift scheduling and management
- **Shift Signups** - User shift registrations
4. **Get Table URLs**
After the script completes:
1. Login to your NocoDB instance at https://db.cmlite.org
2. Navigate to your project ("Map Viewer Project - TIMESTAMP")
3. Copy the view URLs for each table from your browser address bar
4. URLs should look like: `https://db.cmlite.org/dashboard/#/nc/project-id/table-id`
5. **Update Environment with URLs**
Edit your `.env` file and add the table URLs:
```env
NOCODB_VIEW_URL=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/m6g7bkzv7s1w2ur
NOCODB_LOGIN_SHEET=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/mizyc64e4r7ppzh
NOCODB_SETTINGS_SHEET=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/mix06f2mlep7gqb
NOCODB_SHIFTS_SHEET=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/mkx0tex0iquus1u
NOCODB_SHIFT_SIGNUPS_SHEET=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/mi8jg1tn26mu8fj
```
6. **Build and Deploy**
Build the Docker image and start the application:
```bash
# Build the Docker image
docker-compose build
# Start the application
docker-compose up -d
```
7. **Verify Installation**
- Check container status: `docker-compose ps`
- View logs: `docker-compose logs -f map-viewer`
- Access the application at: http://localhost:3000
- Access shift management at: http://localhost:3000/shifts.html
- Access admin panel at: http://localhost:3000/admin.html (admin users only)
## Database Schema
The build script automatically creates the following table structure:
### Main Locations Table
- `ID` (ID): Auto-incrementing primary key
- `Geo-Location` (GeoData): Geographic coordinate data
- `latitude` (Decimal): Precision 8, Scale 8
- `longitude` (Decimal): Precision 8, Scale 8
- `First Name` (Single Line Text): Person's first name
- `Last Name` (Single Line Text): Person's last name
- `Email` (Email): Email address
- `Phone` (PhoneNumber): Phone number with validation
- `Unit Number` (Single Line Text): Unit or apartment number
- `Support Level` (Single Select): Options: "1" (Green), "2" (Yellow), "3" (Orange), "4" (Red)
- `Address` (Single Line Text): Street address
- `Sign` (Checkbox): Has campaign sign
- `Sign Size` (Single Select): Options: "Regular" (Blue), "Large" (Green), "Unsure" (Orange)
- `Notes` (Long Text): Additional details and comments
- `created_by_user` (Single Line Text): Creator email
- `last_updated_by_user` (Single Line Text): Last updater email
### Login Table
- `ID` (ID): Auto-incrementing primary key
- `Email` (Email): User email address (required)
- `Password` (Single Line Text): User password (required)
- `Name` (Single Line Text): User display name
- `Admin` (Checkbox): Admin privileges
- `Created At` (DateTime): Account creation timestamp
- `Last Login` (DateTime): Last login timestamp
### Settings Table
- `ID` (ID): Auto-incrementing primary key
- `created_at` (DateTime): Record creation timestamp
- `created_by` (Single Line Text): Creator identifier
- `Geo-Location` (Single Line Text): Format "latitude;longitude"
- `latitude` (Decimal): Precision 8, Scale 8
- `longitude` (Decimal): Precision 8, Scale 8
- `zoom` (Number): Map zoom level
- `Walk Sheet Title` (Single Line Text): Title for walk sheets
- `Walk Sheet Subtitle` (Single Line Text): Subtitle for walk sheets
- `Walk Sheet Footer` (Long Text): Footer text for walk sheets
- `QR Code 1 URL` (URL): First QR code link
- `QR Code 1 Label` (Single Line Text): First QR code label
- `QR Code 2 URL` (URL): Second QR code link
- `QR Code 2 Label` (Single Line Text): Second QR code label
- `QR Code 3 URL` (URL): Third QR code link
- `QR Code 3 Label` (Single Line Text): Third QR code label
### Shifts Table
- `ID` (ID): Auto-incrementing primary key
- `Title` (Single Line Text): Shift title (required)
- `Description` (Long Text): Detailed shift description
- `Date` (Date): Shift date (required)
- `Start Time` (Time): Shift start time (required)
- `End Time` (Time): Shift end time (required)
- `Location` (Single Line Text): Shift location
- `Max Volunteers` (Number): Maximum volunteer capacity (required)
- `Current Volunteers` (Number): Current volunteer count
- `Status` (Single Select): Options: "Open" (Green), "Full" (Orange), "Cancelled" (Red)
- `Created By` (Single Line Text): Creator identifier
- `Created At` (DateTime): Creation timestamp
- `Updated At` (DateTime): Last update timestamp
### Shift Signups Table
- `ID` (ID): Auto-incrementing primary key
- `Shift ID` (Number): Reference to shifts table (required)
- `Shift Title` (Single Line Text): Copy of shift title for reference
- `User Email` (Email): User's email address (required)
- `User Name` (Single Line Text): User's display name
- `Signup Date` (DateTime): When user signed up
- `Status` (Single Select): Options: "Confirmed" (Green), "Cancelled" (Red)
## API Endpoints
### Public Endpoints
- `GET /api/locations` - Fetch all locations (requires auth)
- `POST /api/locations` - Create new location (requires auth)
- `GET /api/locations/:id` - Get single location (requires auth)
- `PUT /api/locations/:id` - Update location (requires auth)
- `DELETE /api/locations/:id` - Delete location (requires auth)
- `GET /api/config/start-location` - Get map start location
- `GET /health` - Health check
### Shifts Endpoints (requires authentication)
- `GET /api/shifts` - Get all available shifts
- `GET /api/shifts/my-signups` - Get current user's shift signups
- `POST /api/shifts/:shiftId/signup` - Sign up for a shift
- `POST /api/shifts/:shiftId/cancel` - Cancel shift signup
### Shifts Admin Endpoints (requires admin privileges)
- `GET /api/shifts/admin` - Get all shifts including cancelled ones
- `POST /api/shifts/admin` - Create new shift
- `PUT /api/shifts/admin/:id` - Update existing shift
- `DELETE /api/shifts/admin/:id` - Delete shift
### Authentication Endpoints
- `POST /api/auth/login` - User login
- `GET /api/auth/check` - Check authentication status
- `POST /api/auth/logout` - User logout
### Admin Endpoints (requires admin privileges)
- `GET /api/admin/start-location` - Get start location with source info
- `POST /api/admin/start-location` - Update map start location
- `GET /api/admin/walk-sheet-config` - Get walk sheet configuration
- `POST /api/admin/walk-sheet-config` - Save walk sheet configuration
### Geocoding Endpoints (requires authentication)
- `GET /api/geocode/reverse?lat=<lat>&lng=<lng>` - Reverse geocode coordinates to address
- `GET /api/geocode/forward?address=<address>` - Forward geocode address to coordinates
- `GET /api/geocode/search?query=<query>&limit=<number>` - Search for multiple address matches
- `GET /api/geocode/cache/stats` - Get geocoding cache statistics (admin only)
All geocoding endpoints include rate limiting (30 requests per 15 minutes per IP) and support Cloudflare IP detection for accurate rate limiting.
## Shifts Management
The application includes a comprehensive volunteer shift management system accessible at `/shifts.html`.
### User Features
- **Dual View Options**: Toggle between grid view and calendar view for shift display
- **Calendar View**: Interactive monthly calendar showing shifts with color-coded indicators:
- Green: Shifts you've signed up for
- Blue: Available shifts you can join
- Gray: Full shifts (no spots available)
- **View Available Shifts**: See all upcoming shifts with date, time, and capacity information
- **Sign Up for Shifts**: One-click signup for available shifts (works in both views)
- **My Shifts Dashboard**: View all your current shift signups at the top of the page
- **Cancel Signups**: Cancel your shift signups when needed
- **Date Filtering**: Filter shifts by specific dates (applies to both views)
- **Real-time Updates**: Shift availability updates dynamically
- **Interactive Calendar**: Click on calendar shifts for detailed popup with signup options
- **Calendar Navigation**: Navigate between months to view future shifts
### Admin Features
Administrators have additional capabilities for managing shifts:
- **Create New Shifts**: Add new volunteer shifts with date, time, location, and capacity
- **Edit Existing Shifts**: Modify shift details, times, or capacity
- **Cancel Shifts**: Mark shifts as cancelled (they remain in system but hidden from users)
- **View All Signups**: See who has signed up for each shift
- **Manage Capacity**: Set maximum number of volunteers per shift
### Shift Status System
- **Open** (Green): Shift is available and accepting signups
- **Full** (Orange): Shift has reached maximum capacity
- **Cancelled** (Red): Shift has been cancelled by admin
The system automatically updates shift status based on current signups vs. maximum capacity.
## Unified Search System
The application features a powerful unified search system accessible via the search bar in the header or by pressing `Ctrl+K` anywhere in the application.
### Search Modes
The search system operates in two modes:
1. **Documentation Search**: Search through integrated MkDocs documentation
2. **Address Search**: Search for addresses and geographic locations
### Features
- **Mode Toggle**: Switch between docs and address search with dedicated buttons
- **Keyboard Shortcuts**:
- `Ctrl+K` or `Cmd+K`: Focus search input from anywhere
- `Escape`: Close search results
- Arrow keys: Navigate through search results
- `Enter`: Select highlighted result
- **Real-time Results**: Search results update as you type (minimum 2 characters)
- **Smart Caching**: Results are cached for improved performance
- **QR Code Generation**: Generate QR codes for documentation links
- **Visual Feedback**: Loading states, result counts, and error handling
### Documentation Search
When connected to a MkDocs documentation site:
- **Full-text Search**: Search through all documentation content
- **Snippet Preview**: See relevant excerpts with search terms highlighted
- **Direct Navigation**: Click results to open documentation pages
- **Path Display**: Shows the document path and section
- **QR Code Support**: Generate QR codes for sharing documentation links
### Address Search
For geographic location search:
- **Geocoding Integration**: Powered by Nominatim/OpenStreetMap
- **Multiple Results**: Returns up to 5 address matches
- **Map Integration**: Click results to view location on map
- **Temporary Markers**: Visual markers for search results
- **Quick Actions**: Add locations directly from search results
- **Coordinate Display**: Shows precise latitude/longitude coordinates
### Configuration
The unified search system integrates with MkDocs documentation when configured:
```env
MKDOCS_URL=https://your-docs-site.com
MKDOCS_SEARCH_URL=https://your-docs-site.com
MKDOCS_SITE_SERVER_PORT=4002
```
### Rate Limiting
Address search is rate-limited to prevent abuse:
- 30 requests per 15-minute window per IP
- Cloudflare IP detection for accurate limiting
- Graceful error handling for rate limit exceeded
## Admin Panel
## Unified Search System
The application features a powerful unified search system accessible via the search bar in the header.
<!-- In the Unified Search System section, update the Search Modes: -->
### Search Modes
The search system operates in three modes:
1. **Documentation Search**: Search through integrated MkDocs documentation
2. **Address Search**: Search for addresses and geographic locations
3. **Database Search**: Search through loaded location records on the map
### Features
- **Mode Toggle**: Switch between docs, address, and database search with dedicated buttons
- **Keyboard Shortcuts**:
- `Ctrl+K` or `Cmd+K`: Focus search input from anywhere
- `Ctrl+Shift+D`: Switch to docs mode
- `Ctrl+Shift+M`: Switch to map mode
- `Ctrl+Shift+B`: Switch to database mode
- `Escape`: Close search results
- Arrow keys: Navigate through search results
- `Enter`: Select highlighted result
### Database Search
For searching through loaded location data:
- **Full-text Search**: Search through names, addresses, emails, phone numbers, and notes
- **Smart Matching**: Finds partial matches across multiple fields
- **Result Preview**: See relevant details with search terms highlighted
- **Map Integration**: Click results to pan to location and open marker popup
- **Marker Highlighting**: Temporarily highlights selected markers on the map
- **Fast Performance**: Searches through already-loaded data for instant results
## Admin Panel
Users with admin privileges can access the admin panel at `/admin.html` to configure system settings.
### Features
#### Dashboard Analytics
- **Campaign Overview**: Real-time statistics and metrics
- **Support Distribution**: Visual breakdown of support levels (1-4)
- **Sign Tracking**: Monitor lawn sign requests
- **User Analytics**: Track user growth and daily entries
- **Performance Score**: Overall campaign performance metric
#### Start Location Configuration
- **Interactive Map**: Visual interface for selecting coordinates
- **Real-time Preview**: See changes immediately on the admin map
- **Validation**: Built-in coordinate and zoom level validation
#### Walk Sheet Generator
- **Printable Forms**: Generate 8.5x11 walk sheets for door-to-door canvassing
- **QR Code Integration**: Add up to 3 QR codes with custom URLs and labels
- **Form Field Matching**: Automatically matches fields from the main location form
- **Live Preview**: See changes as you type
- **Print Optimization**: Proper formatting for printing or PDF export
- **Persistent Storage**: All QR codes and settings saved to NocoDB
#### Shift Management
- **Create Shifts**: Set up volunteer shifts with dates, times, and capacity
- **Manage Volunteers**: View signups and manage shift participants
- **Real-time Updates**: See shift status changes immediately
#### User Management
- **Create Users**: Add new user accounts with email and password
- **Role Assignment**: Assign admin or user privileges
- **User List**: View all registered users with their details and creation dates
- **Delete Users**: Remove user accounts (with confirmation prompts)
- **Security**: Password validation and admin-only access
#### Convert Data
- **CSV Upload**: Upload CSV files containing addresses for bulk import
- **Drag & Drop Interface**: Easy file upload with visual feedback
- **Real-time Geocoding**: Addresses are geocoded in real-time with progress tracking
- **Visual Progress**: Live progress bar and status updates during processing
- **Map Preview**: Interactive map showing geocoded locations before saving
- **Results Table**: Detailed table with success/error status for each address
- **Batch Save**: Save all successfully geocoded locations to the database
- **Field Mapping**: Automatically maps common CSV fields (First Name, Last Name, Email, Phone, etc.)
- **Error Handling**: Clear error messages for failed geocoding attempts
- **File Validation**: CSV format validation and file size limits (10MB max)
### Access Control
- Admin access is controlled via the `Admin` checkbox in the Login table
- Only authenticated users with admin privileges can access `/admin.html`
- Admin status is checked on every request to admin endpoints
- User management functions are restricted to admin users only
### Start Location Priority
The system uses a cascading fallback system for map start location:
1. **Database**: Admin-configured location stored in Settings table (highest priority)
2. **Environment**: Default values from .env file (medium priority)
3. **Hardcoded**: Edmonton, Canada coordinates (lowest priority)
## Configuration
All configuration is done via environment variables:
| Variable | Description | Default |
|----------|-------------|---------|
| `NOCODB_API_URL` | NocoDB API base URL | Required |
| `NOCODB_API_TOKEN` | API authentication token | Required |
| `NOCODB_VIEW_URL` | Full NocoDB view URL for locations table | Required |
| `NOCODB_LOGIN_SHEET` | Login table URL for authentication | Required |
| `NOCODB_SETTINGS_SHEET` | Settings table URL for admin config | Required |
| `NOCODB_SHIFTS_SHEET` | Shifts table URL for shift management | Required |
| `NOCODB_SHIFT_SIGNUPS_SHEET` | Shift signups table URL for user registrations | Required |
| `DOMAIN` | Primary domain for the application | Required |
| `MKDOCS_URL` | MkDocs documentation site URL | Optional |
| `MKDOCS_SEARCH_URL` | MkDocs search endpoint URL | Optional |
| `MKDOCS_SITE_SERVER_PORT` | Port for MkDocs integration | 4002 |
| `PORT` | Server port | 3000 |
| `NODE_ENV` | Environment mode | production |
| `SESSION_SECRET` | Session encryption secret (generate with openssl rand -hex 32) | Required |
| `DEFAULT_LAT` | Default map latitude | 53.5461 |
| `DEFAULT_LNG` | Default map longitude | -113.4938 |
| `DEFAULT_ZOOM` | Default map zoom level | 11 |
| `BOUND_NORTH` | Northern boundary for map points (optional) | None |
| `BOUND_SOUTH` | Southern boundary for map points (optional) | None |
| `BOUND_EAST` | Eastern boundary for map points (optional) | None |
| `BOUND_WEST` | Western boundary for map points (optional) | None |
| `TRUST_PROXY` | Trust proxy headers (for Cloudflare) | true |
| `COOKIE_DOMAIN` | Cookie domain setting | .cmlite.org |
| `ALLOWED_ORIGINS` | CORS allowed origins (comma-separated) | Multiple URLs |
| `SMTP_HOST` | SMTP server hostname (optional) | smtp.gmail.com |
| `SMTP_PORT` | SMTP server port (optional) | 587 |
| `SMTP_SECURE` | Use SSL for SMTP (optional) | false |
| `SMTP_USER` | SMTP username (optional) | your-email@gmail.com |
| `SMTP_PASS` | SMTP password (optional) | your-app-password |
| `EMAIL_FROM_NAME` | Email sender name (optional) | CMlite Support |
| `EMAIL_FROM_ADDRESS` | Email sender address (optional) | noreply@cmlite.org |
| `APP_NAME` | Application name for emails (optional) | CMlite Map |
## Maintenance Commands
### Update Application
```bash
docker-compose down
git pull origin main
docker-compose build
docker-compose up -d
```
### Development Mode
```bash
cd app
npm install
npm run dev
```
### Health Check
```bash
curl http://localhost:3000/health
```
## Development
To run in development mode:
1. Install dependencies:
```bash
cd app
npm install
```
2. Start with hot reload:
```bash
npm run dev
```
## Security Considerations
- API tokens are kept server-side only
- CORS is configured for security
- Rate limiting prevents abuse
- Input validation on all endpoints
- Helmet.js for security headers
## Troubleshooting
### Locations not showing
- Verify table has `Geo-Location`, `latitude`, and `longitude` columns
- Check that coordinates are valid numbers
- Ensure API token has read permissions
### Cannot add locations
- Verify API token has write permissions
- Check browser console for errors
- Ensure coordinates are within valid ranges
### Connection errors
- Verify NocoDB instance is accessible
- Check API URL format
- Confirm network connectivity
### Build Script Issues
- Ensure NocoDB instance is accessible
- Verify API token has admin permissions
- Check that the NocoDB database is clean (delete all bases before running)
## License
MIT License - See LICENSE file for details
## Support
For issues or questions:
1. Check the troubleshooting section
2. Review NocoDB documentation
3. Open an issue on GitHub
## Known Bugs
- First load of page often fails, need to debug
- Want UI for dots to have an edit button that then brings up the form