admin 5b673dacc2 A tonne more changes, including new nocodb admin section, search for database, code cleanups, and debugging

2025-08-01 15:01:35 -06:00

22 KiB

Raw Blame History

NocoDB Map Viewer

A containerized web application that visualizes geographic data from NocoDB on an interactive map using Leaflet.js.

Features

🗺️ Interactive map visualization with OpenStreetMap
🔍 Unified search system with docs and address search (Ctrl+K to activate)
📍 Real-time geolocation support
➕ Add new locations directly from the map
🔄 Auto-refresh every 30 seconds
📱 Responsive design for mobile devices
🔒 Secure API proxy to protect credentials
👤 User authentication with login system
⚙️ Admin panel for system configuration
🎯 Configurable map start location
📋 Walk Sheet generator for door-to-door canvassing
🔗 QR code integration for digital resources
📅 Volunteer shift management system with calendar and grid views
✋ User shift signup and cancellation with color-coded calendar
👥 Admin shift creation and management
👨‍💼 User management panel for admin users (create, delete users)
🔐 Role-based access control (Admin vs User permissions)
📧 Email notifications and password recovery via SMTP
📊 CSV data import with batch geocoding and visual progress tracking
🐳 Docker containerization for easy deployment
🆓 100% open source (no proprietary dependencies)

Quick Start

Prerequisites

Docker and Docker Compose
NocoDB instance with API access
NocoDB API token

Installation

Get NocoDB API Token
1. Login to your NocoDB instance
2. Click user icon → Account Settings → API Tokens
3. Create new token with read/write permissions
4. Copy the token for the next step

Configure Environment

Edit the .env file with your NocoDB API and API Url:

# NocoDB API Configuration
NOCODB_API_URL=https://db.cmlite.org/api/v1
NOCODB_API_TOKEN=your-api-token-here

# These will be populated after running build-nocodb.sh
NOCODB_VIEW_URL=
NOCODB_LOGIN_SHEET=
NOCODB_SETTINGS_SHEET=
NOCODB_SHIFTS_SHEET=
NOCODB_SHIFT_SIGNUPS_SHEET=

# Domain Configuration
DOMAIN=cmlite.org

# MkDocs Integration
MKDOCS_URL=https://cmlite.org
MKDOCS_SEARCH_URL=https://cmlite.org
MKDOCS_SITE_SERVER_PORT=4002

# Server Configuration
PORT=3000
NODE_ENV=production

# Session Secret (Generate with: openssl rand -hex 32)
SESSION_SECRET=your-secure-random-string

# Map Defaults (Edmonton, Alberta, Canada)
DEFAULT_LAT=53.5461
DEFAULT_LNG=-113.4938
DEFAULT_ZOOM=11

# Optional: Map Boundaries (prevents users from adding points outside area)
# BOUND_NORTH=53.7
# BOUND_SOUTH=53.4
# BOUND_EAST=-113.3
# BOUND_WEST=-113.7

# Cloudflare Settings
TRUST_PROXY=true
COOKIE_DOMAIN=.cmlite.org

# Allowed Origins
ALLOWED_ORIGINS=https://map.cmlite.org,http://localhost:3000

# Email Configuration (Optional - for password recovery)
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_SECURE=false
SMTP_USER=your-email@gmail.com
SMTP_PASS=your-app-password
EMAIL_FROM_NAME=CMlite Support
EMAIL_FROM_ADDRESS=noreply@cmlite.org
APP_NAME=CMlite Map

Auto-Create Database Structure

Run the build script to create required tables:
```
chmod +x build-nocodb.sh
./build-nocodb.sh
```
This creates five tables:
- Locations - Main map data with geo-location, contact info, support levels
- Login - User authentication (email, name, admin flag)
- Settings - Admin configuration and QR codes
- Shifts - Shift scheduling and management
- Shift Signups - User shift registrations
Get Table URLs

After the script completes:
1. Login to your NocoDB instance at https://db.cmlite.org
2. Navigate to your project ("Map Viewer Project - TIMESTAMP")
3. Copy the view URLs for each table from your browser address bar
4. URLs should look like: https://db.cmlite.org/dashboard/#/nc/project-id/table-id

Update Environment with URLs

Edit your .env file and add the table URLs:

NOCODB_VIEW_URL=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/m6g7bkzv7s1w2ur
NOCODB_LOGIN_SHEET=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/mizyc64e4r7ppzh
NOCODB_SETTINGS_SHEET=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/mix06f2mlep7gqb
NOCODB_SHIFTS_SHEET=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/mkx0tex0iquus1u
NOCODB_SHIFT_SIGNUPS_SHEET=https://db.cmlite.org/dashboard/#/nc/pnsalzrup2zqvz8/mi8jg1tn26mu8fj

Build and Deploy

Build the Docker image and start the application:

# Build the Docker image
docker-compose build

# Start the application
docker-compose up -d

Verify Installation
- Check container status: docker-compose ps
- View logs: docker-compose logs -f map-viewer
- Access the application at: http://localhost:3000
- Access shift management at: http://localhost:3000/shifts.html
- Access admin panel at: http://localhost:3000/admin.html (admin users only)

Database Schema

The build script automatically creates the following table structure:

Main Locations Table

ID (ID): Auto-incrementing primary key
Geo-Location (GeoData): Geographic coordinate data
latitude (Decimal): Precision 8, Scale 8
longitude (Decimal): Precision 8, Scale 8
First Name (Single Line Text): Person's first name
Last Name (Single Line Text): Person's last name
Email (Email): Email address
Phone (PhoneNumber): Phone number with validation
Unit Number (Single Line Text): Unit or apartment number
Support Level (Single Select): Options: "1" (Green), "2" (Yellow), "3" (Orange), "4" (Red)
Address (Single Line Text): Street address
Sign (Checkbox): Has campaign sign
Sign Size (Single Select): Options: "Regular" (Blue), "Large" (Green), "Unsure" (Orange)
Notes (Long Text): Additional details and comments
created_by_user (Single Line Text): Creator email
last_updated_by_user (Single Line Text): Last updater email

ID (ID): Auto-incrementing primary key
Email (Email): User email address (required)
Password (Single Line Text): User password (required)
Name (Single Line Text): User display name
Admin (Checkbox): Admin privileges
Created At (DateTime): Account creation timestamp
Last Login (DateTime): Last login timestamp

Settings Table

ID (ID): Auto-incrementing primary key
created_at (DateTime): Record creation timestamp
created_by (Single Line Text): Creator identifier
Geo-Location (Single Line Text): Format "latitude;longitude"
latitude (Decimal): Precision 8, Scale 8
longitude (Decimal): Precision 8, Scale 8
zoom (Number): Map zoom level
Walk Sheet Title (Single Line Text): Title for walk sheets
Walk Sheet Subtitle (Single Line Text): Subtitle for walk sheets
Walk Sheet Footer (Long Text): Footer text for walk sheets
QR Code 1 URL (URL): First QR code link
QR Code 1 Label (Single Line Text): First QR code label
QR Code 2 URL (URL): Second QR code link
QR Code 2 Label (Single Line Text): Second QR code label
QR Code 3 URL (URL): Third QR code link
QR Code 3 Label (Single Line Text): Third QR code label

Shifts Table

ID (ID): Auto-incrementing primary key
Title (Single Line Text): Shift title (required)
Description (Long Text): Detailed shift description
Date (Date): Shift date (required)
Start Time (Time): Shift start time (required)
End Time (Time): Shift end time (required)
Location (Single Line Text): Shift location
Max Volunteers (Number): Maximum volunteer capacity (required)
Current Volunteers (Number): Current volunteer count
Status (Single Select): Options: "Open" (Green), "Full" (Orange), "Cancelled" (Red)
Created By (Single Line Text): Creator identifier
Created At (DateTime): Creation timestamp
Updated At (DateTime): Last update timestamp

Shift Signups Table

ID (ID): Auto-incrementing primary key
Shift ID (Number): Reference to shifts table (required)
Shift Title (Single Line Text): Copy of shift title for reference
User Email (Email): User's email address (required)
User Name (Single Line Text): User's display name
Signup Date (DateTime): When user signed up
Status (Single Select): Options: "Confirmed" (Green), "Cancelled" (Red)

API Endpoints

Public Endpoints

GET /api/locations - Fetch all locations (requires auth)
POST /api/locations - Create new location (requires auth)
GET /api/locations/:id - Get single location (requires auth)
PUT /api/locations/:id - Update location (requires auth)
DELETE /api/locations/:id - Delete location (requires auth)
GET /api/config/start-location - Get map start location
GET /health - Health check

Shifts Endpoints (requires authentication)

GET /api/shifts - Get all available shifts
GET /api/shifts/my-signups - Get current user's shift signups
POST /api/shifts/:shiftId/signup - Sign up for a shift
POST /api/shifts/:shiftId/cancel - Cancel shift signup

Shifts Admin Endpoints (requires admin privileges)

GET /api/shifts/admin - Get all shifts including cancelled ones
POST /api/shifts/admin - Create new shift
PUT /api/shifts/admin/:id - Update existing shift
DELETE /api/shifts/admin/:id - Delete shift

Authentication Endpoints

POST /api/auth/login - User login
GET /api/auth/check - Check authentication status
POST /api/auth/logout - User logout

Admin Endpoints (requires admin privileges)

GET /api/admin/start-location - Get start location with source info
POST /api/admin/start-location - Update map start location
GET /api/admin/walk-sheet-config - Get walk sheet configuration
POST /api/admin/walk-sheet-config - Save walk sheet configuration

Geocoding Endpoints (requires authentication)

GET /api/geocode/reverse?lat=<lat>&lng=<lng> - Reverse geocode coordinates to address
GET /api/geocode/forward?address=<address> - Forward geocode address to coordinates
GET /api/geocode/search?query=<query>&limit=<number> - Search for multiple address matches
GET /api/geocode/cache/stats - Get geocoding cache statistics (admin only)

All geocoding endpoints include rate limiting (30 requests per 15 minutes per IP) and support Cloudflare IP detection for accurate rate limiting.

Shifts Management

The application includes a comprehensive volunteer shift management system accessible at /shifts.html.

User Features

Dual View Options: Toggle between grid view and calendar view for shift display
Calendar View: Interactive monthly calendar showing shifts with color-coded indicators:
- Green: Shifts you've signed up for
- Blue: Available shifts you can join
- Gray: Full shifts (no spots available)
View Available Shifts: See all upcoming shifts with date, time, and capacity information
Sign Up for Shifts: One-click signup for available shifts (works in both views)
My Shifts Dashboard: View all your current shift signups at the top of the page
Cancel Signups: Cancel your shift signups when needed
Date Filtering: Filter shifts by specific dates (applies to both views)
Real-time Updates: Shift availability updates dynamically
Interactive Calendar: Click on calendar shifts for detailed popup with signup options
Calendar Navigation: Navigate between months to view future shifts

Admin Features

Administrators have additional capabilities for managing shifts:

Create New Shifts: Add new volunteer shifts with date, time, location, and capacity
Edit Existing Shifts: Modify shift details, times, or capacity
Cancel Shifts: Mark shifts as cancelled (they remain in system but hidden from users)
View All Signups: See who has signed up for each shift
Manage Capacity: Set maximum number of volunteers per shift

Shift Status System

Open (Green): Shift is available and accepting signups
Full (Orange): Shift has reached maximum capacity
Cancelled (Red): Shift has been cancelled by admin

The system automatically updates shift status based on current signups vs. maximum capacity.

Unified Search System

The application features a powerful unified search system accessible via the search bar in the header or by pressing Ctrl+K anywhere in the application.

Search Modes

The search system operates in two modes:

Documentation Search: Search through integrated MkDocs documentation
Address Search: Search for addresses and geographic locations

Features

Mode Toggle: Switch between docs and address search with dedicated buttons
Keyboard Shortcuts:
- Ctrl+K or Cmd+K: Focus search input from anywhere
- Escape: Close search results
- Arrow keys: Navigate through search results
- Enter: Select highlighted result
Real-time Results: Search results update as you type (minimum 2 characters)
Smart Caching: Results are cached for improved performance
QR Code Generation: Generate QR codes for documentation links
Visual Feedback: Loading states, result counts, and error handling

Documentation Search

When connected to a MkDocs documentation site:

Full-text Search: Search through all documentation content
Snippet Preview: See relevant excerpts with search terms highlighted
Direct Navigation: Click results to open documentation pages
Path Display: Shows the document path and section
QR Code Support: Generate QR codes for sharing documentation links

Address Search

For geographic location search:

Geocoding Integration: Powered by Nominatim/OpenStreetMap
Multiple Results: Returns up to 5 address matches
Map Integration: Click results to view location on map
Temporary Markers: Visual markers for search results
Quick Actions: Add locations directly from search results
Coordinate Display: Shows precise latitude/longitude coordinates

Configuration

The unified search system integrates with MkDocs documentation when configured:

MKDOCS_URL=https://your-docs-site.com
MKDOCS_SEARCH_URL=https://your-docs-site.com
MKDOCS_SITE_SERVER_PORT=4002

Rate Limiting

Address search is rate-limited to prevent abuse:

30 requests per 15-minute window per IP
Cloudflare IP detection for accurate limiting
Graceful error handling for rate limit exceeded

Admin Panel

Unified Search System

The application features a powerful unified search system accessible via the search bar in the header.

Search Modes

The search system operates in three modes:

Documentation Search: Search through integrated MkDocs documentation
Address Search: Search for addresses and geographic locations
Database Search: Search through loaded location records on the map

Features

Mode Toggle: Switch between docs, address, and database search with dedicated buttons
Keyboard Shortcuts:
- Ctrl+K or Cmd+K: Focus search input from anywhere
- Ctrl+Shift+D: Switch to docs mode
- Ctrl+Shift+M: Switch to map mode
- Ctrl+Shift+B: Switch to database mode
- Escape: Close search results
- Arrow keys: Navigate through search results
- Enter: Select highlighted result

Database Search

For searching through loaded location data:

Full-text Search: Search through names, addresses, emails, phone numbers, and notes
Smart Matching: Finds partial matches across multiple fields
Result Preview: See relevant details with search terms highlighted
Map Integration: Click results to pan to location and open marker popup
Marker Highlighting: Temporarily highlights selected markers on the map
Fast Performance: Searches through already-loaded data for instant results

Admin Panel

Users with admin privileges can access the admin panel at /admin.html to configure system settings.

Features

Dashboard Analytics

Campaign Overview: Real-time statistics and metrics
Support Distribution: Visual breakdown of support levels (1-4)
Sign Tracking: Monitor lawn sign requests
User Analytics: Track user growth and daily entries
Performance Score: Overall campaign performance metric

Start Location Configuration

Interactive Map: Visual interface for selecting coordinates
Real-time Preview: See changes immediately on the admin map
Validation: Built-in coordinate and zoom level validation

Walk Sheet Generator

Printable Forms: Generate 8.5x11 walk sheets for door-to-door canvassing
QR Code Integration: Add up to 3 QR codes with custom URLs and labels
Form Field Matching: Automatically matches fields from the main location form
Live Preview: See changes as you type
Print Optimization: Proper formatting for printing or PDF export
Persistent Storage: All QR codes and settings saved to NocoDB

Shift Management

Create Shifts: Set up volunteer shifts with dates, times, and capacity
Manage Volunteers: View signups and manage shift participants
Real-time Updates: See shift status changes immediately

User Management

Create Users: Add new user accounts with email and password
Role Assignment: Assign admin or user privileges
User List: View all registered users with their details and creation dates
Delete Users: Remove user accounts (with confirmation prompts)
Security: Password validation and admin-only access

Convert Data

CSV Upload: Upload CSV files containing addresses for bulk import
Drag & Drop Interface: Easy file upload with visual feedback
Real-time Geocoding: Addresses are geocoded in real-time with progress tracking
Visual Progress: Live progress bar and status updates during processing
Map Preview: Interactive map showing geocoded locations before saving
Results Table: Detailed table with success/error status for each address
Batch Save: Save all successfully geocoded locations to the database
Field Mapping: Automatically maps common CSV fields (First Name, Last Name, Email, Phone, etc.)
Error Handling: Clear error messages for failed geocoding attempts
File Validation: CSV format validation and file size limits (10MB max)

Access Control

Admin access is controlled via the Admin checkbox in the Login table
Only authenticated users with admin privileges can access /admin.html
Admin status is checked on every request to admin endpoints
User management functions are restricted to admin users only

Start Location Priority

The system uses a cascading fallback system for map start location:

Database: Admin-configured location stored in Settings table (highest priority)
Environment: Default values from .env file (medium priority)
Hardcoded: Edmonton, Canada coordinates (lowest priority)

Configuration

All configuration is done via environment variables:

Variable	Description	Default
`NOCODB_API_URL`	NocoDB API base URL	Required
`NOCODB_API_TOKEN`	API authentication token	Required
`NOCODB_VIEW_URL`	Full NocoDB view URL for locations table	Required
`NOCODB_LOGIN_SHEET`	Login table URL for authentication	Required
`NOCODB_SETTINGS_SHEET`	Settings table URL for admin config	Required
`NOCODB_SHIFTS_SHEET`	Shifts table URL for shift management	Required
`NOCODB_SHIFT_SIGNUPS_SHEET`	Shift signups table URL for user registrations	Required
`DOMAIN`	Primary domain for the application	Required
`MKDOCS_URL`	MkDocs documentation site URL	Optional
`MKDOCS_SEARCH_URL`	MkDocs search endpoint URL	Optional
`MKDOCS_SITE_SERVER_PORT`	Port for MkDocs integration	4002
`PORT`	Server port	3000
`NODE_ENV`	Environment mode	production
`SESSION_SECRET`	Session encryption secret (generate with openssl rand -hex 32)	Required
`DEFAULT_LAT`	Default map latitude	53.5461
`DEFAULT_LNG`	Default map longitude	-113.4938
`DEFAULT_ZOOM`	Default map zoom level	11
`BOUND_NORTH`	Northern boundary for map points (optional)	None
`BOUND_SOUTH`	Southern boundary for map points (optional)	None
`BOUND_EAST`	Eastern boundary for map points (optional)	None
`BOUND_WEST`	Western boundary for map points (optional)	None
`TRUST_PROXY`	Trust proxy headers (for Cloudflare)	true
`COOKIE_DOMAIN`	Cookie domain setting	.cmlite.org
`ALLOWED_ORIGINS`	CORS allowed origins (comma-separated)	Multiple URLs
`SMTP_HOST`	SMTP server hostname (optional)	smtp.gmail.com
`SMTP_PORT`	SMTP server port (optional)	587
`SMTP_SECURE`	Use SSL for SMTP (optional)	false
`SMTP_USER`	SMTP username (optional)	your-email@gmail.com
`SMTP_PASS`	SMTP password (optional)	your-app-password
`EMAIL_FROM_NAME`	Email sender name (optional)	CMlite Support
`EMAIL_FROM_ADDRESS`	Email sender address (optional)	noreply@cmlite.org
`APP_NAME`	Application name for emails (optional)	CMlite Map

Maintenance Commands

Update Application

docker-compose down
git pull origin main
docker-compose build
docker-compose up -d

Development Mode

cd app
npm install
npm run dev

Health Check

curl http://localhost:3000/health

Development

To run in development mode:

Install dependencies:
```
cd app
npm install
```
Start with hot reload:
```
npm run dev
```

Security Considerations

API tokens are kept server-side only
CORS is configured for security
Rate limiting prevents abuse
Input validation on all endpoints
Helmet.js for security headers

Troubleshooting

Locations not showing

Verify table has Geo-Location, latitude, and longitude columns
Check that coordinates are valid numbers
Ensure API token has read permissions

Cannot add locations

Verify API token has write permissions
Check browser console for errors
Ensure coordinates are within valid ranges

Connection errors

Verify NocoDB instance is accessible
Check API URL format
Confirm network connectivity

Build Script Issues

Ensure NocoDB instance is accessible
Verify API token has admin permissions
Check that the NocoDB database is clean (delete all bases before running)

License

MIT License - See LICENSE file for details

Support

For issues or questions:

Check the troubleshooting section
Review NocoDB documentation
Open an issue on GitHub

Known Bugs

First load of page often fails, need to debug
Want UI for dots to have an edit button that then brings up the form

22 KiB Raw Blame History Unescape Escape

NocoDB Map Viewer

Features

Quick Start

Prerequisites

Installation

Database Schema

Main Locations Table

Login Table

Settings Table

Shifts Table

Shift Signups Table

API Endpoints

Public Endpoints

Shifts Endpoints (requires authentication)

Shifts Admin Endpoints (requires admin privileges)

Authentication Endpoints

Admin Endpoints (requires admin privileges)

Geocoding Endpoints (requires authentication)

Shifts Management

User Features

Admin Features

Shift Status System

Unified Search System

Search Modes

Features

Documentation Search

Address Search

Configuration

Rate Limiting

Admin Panel

Unified Search System

Search Modes

Features

Database Search

Admin Panel

Features

Dashboard Analytics

Start Location Configuration

Walk Sheet Generator

Shift Management

User Management

Convert Data

Access Control

Start Location Priority

Configuration

Maintenance Commands

Update Application

Development Mode

Health Check

Development

Security Considerations

Troubleshooting

Locations not showing

Cannot add locations

Connection errors

Build Script Issues

License

Support

Known Bugs

22 KiB

Raw Blame History