Ingress Intel Report

A comprehensive system for fetching, analyzing, and storing game events from the Ingress Intel API.

Overview

The Ingress Intel Report is a multi-component system consisting of:

• CLI Tool (main.py) - Command-line interface for querying events
• Web API (app.py) - REST API for programmatic access
• Scheduler (schedule.py) - Automated data collection every minute
• MongoDB - Persistent storage for historical data
• Docker - Containerized deployment with docker-compose

Features

• Geographic Filtering: Query events within a bounding box
• Event Classification: Automatic event type detection (portal captures, links, resonators, etc.)
• Player Tracking: Filter events by specific players
• Timestamp Filtering: Filter events by time range
• Coordinate Extraction: Extract portal locations from events
• REST API: JSON-based API for programmatic access
• MongoDB Persistence: Store historical event data
• Scheduled Collection: Automated data collection every minute
• Docker Deployment: Containerized deployment with health checks
• Timezone Support: Configurable timezone (default: Europe/Rome)

Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   CLI Tool  │     │  Web API    │     │  Scheduler  │
│  (main.py)  │     │  (app.py)   │     │(schedule.py)│
└──────┬──────┘     └──────┬──────┘     └──────┬──────┘
       │                   │                   │
       └───────────────────┼───────────────────┘
                           │
                    ┌──────▼──────┐
                    │ Ingress API │
                    │ (ingress.py)│
                    └──────┬──────┘
                           │
                    ┌──────▼──────┐
                    │  MongoDB    │
                    │  Database   │
                    └─────────────┘

Installation

Option 1: Docker (Recommended)

The easiest way to run the complete system is using Docker Compose:

# Clone the repository
git clone <repository-url>
cd Ingress

# Create environment file
cp .env.example .env
# Edit .env with your configuration

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f

# Stop services
docker-compose down

Option 2: Local Development

For local development without Docker:

# Install dependencies
pip install -r requirements.txt

# Or using uv (recommended)
uv sync

# Set up MongoDB
# Install MongoDB locally or use a cloud instance
# Update MONGO_URI in .env file

# Start the web API
python app.py

# In another terminal, start the scheduler
python schedule.py

Option 3: CLI Only

If you only need the CLI tool:

# Install dependencies
pip install -r requirements.txt

# Set environment variables
export INGRESS_COOKIE="your_cookie_here"
export V="your_api_version"

# Run the CLI
python main.py --help

Configuration

Create a .env file in the project root with the following variables:

Variable	Required	Default	Description
INGRESS_COOKIE	Yes	-	Ingress authentication cookie (includes csrftoken and sessionid)
V	Yes	-	Ingress API version string
PORT	No	5000	Web API server port
MONGO_URI	Yes*	-	MongoDB connection string (required for scheduler)
DB_NAME	Yes*	-	MongoDB database name (required for scheduler)
COLLECTION_NAME	Yes*	-	MongoDB collection name (required for scheduler)
ENDPOINT_URL	Yes*	-	API endpoint for scheduler (e.g., http://web:7000/plexts)
TZ	No	Europe/Rome	Timezone for timestamps
MIN_LAT	No	45470259	Default minimum latitude (E6 format)
MIN_LNG	No	12244155	Default minimum longitude (E6 format)
MAX_LAT	No	45480370	Default maximum latitude (E6 format)
MAX_LNG	No	12298207	Default maximum longitude (E6 format)

*Required for scheduler and web API with MongoDB integration

Example .env file

# Ingress API Configuration
INGRESS_COOKIE="csrftoken=your_token; sessionid=your_session"
V="412c0ac7e784d6df783fc507bca30e23b3c58c55"

# Web API Configuration
PORT=7000

# MongoDB Configuration
MONGO_URI="mongodb://ingress_root:password@localhost:27017/"
DB_NAME=ingress
COLLECTION_NAME=ingress

# Scheduler Configuration
ENDPOINT_URL="http://localhost:7000/plexts"

# Timezone
TZ=Europe/Rome

# Geographic Bounds (Venice, Italy)
MIN_LAT=45470259
MIN_LNG=12244155
MAX_LAT=45480370
MAX_LNG=12298207

Usage

CLI Tool

The CLI tool provides a command-line interface for querying Ingress events.

Basic Usage

# Get all events in default area (Venice, Italy)
python main.py

# Show help
python main.py --help

Command Line Options

Option	Type	Default	Description
--event-types	Multiple strings	None	Filter by event types (comma-separated or space-separated)
--player-name	String	None	Filter events by a specific player name
--min-lat	Integer	45470259	Minimum latitude (in microdegrees, E6 format)
--min-lng	Integer	12244155	Minimum longitude (in microdegrees, E6 format)
--max-lat	Integer	45480370	Maximum latitude (in microdegrees, E6 format)
--max-lng	Integer	12298207	Maximum longitude (in microdegrees, E6 format)
--min-timestamp	Timestamp	-1	Minimum timestamp (milliseconds since epoch or ISO 8601 format)
--max-timestamp	Timestamp	-1	Maximum timestamp (milliseconds since epoch or ISO 8601 format)

Timestamp Filtering

The --min-timestamp and --max-timestamp options support two formats:

Format 1: Milliseconds (integer)

python main.py --min-timestamp 1736659200000 --max-timestamp 1736745600000

Format 2: ISO 8601 datetime string

python main.py --min-timestamp "2026-01-12T00:00:00Z" --max-timestamp "2026-01-13T00:00:00Z"

Examples:

# Get events from last hour
python main.py --min-timestamp 1736659200000

# Get events for a specific day
python main.py --min-timestamp "2026-01-12T00:00:00Z" --max-timestamp "2026-01-13T00:00:00Z"

# Get events from a specific time onwards
python main.py --min-timestamp "2026-01-12T10:00:00Z"

# Get events up to a specific time
python main.py --max-timestamp "2026-01-12T12:00:00Z"

Event Type Filtering

# Filter by single event type
python main.py --event-types PORTAL_CAPTURED

# Filter by multiple event types (space-separated)
python main.py --event-types PORTAL_CAPTURED LINK_CREATED CONTROL_FIELD_CREATED

# Filter by resonator events
python main.py --event-types RESONATOR_DEPLOYED RESONATOR_DESTROYED

Player Filtering

# Filter by player name
python main.py --player-name Albicocca

# Combine with event type filter
python main.py --player-name Albicocca --event-types PORTAL_CAPTURED

Geographic Filtering

Coordinates are in E6 format (microdegrees): multiply decimal degrees by 1,000,000.

# Custom geographic bounds
python main.py --min-lat 45470000 --max-lat 45480000 --min-lng 12240000 --max-lng 12300000

Combining Filters

# Complex query with multiple filters
python main.py \
  --min-timestamp "2026-01-12T10:00:00Z" \
  --max-timestamp "2026-01-12T12:00:00Z" \
  --event-types PORTAL_CAPTURED LINK_CREATED \
  --player-name Albicocca \
  --min-lat 45470000 \
  --max-lat 45480000

Web API

The web API provides a REST interface for programmatic access to Ingress events.

Starting the Web API

# Development mode
python app.py

# Production mode with Gunicorn
gunicorn -w 4 -b 0.0.0.0:7000 app:app

# Or using the entrypoint script
./entrypoint-web.sh

API Endpoints

GET /

Returns API information and available endpoints.

Response:

{
  "name": "Ingress Intel API",
  "version": "1.0.0",
  "endpoints": {
    "/plexts": {
      "method": "GET",
      "description": "Get plexts from the Ingress API",
      "parameters": {
        "event_types": "List of event types to filter by (comma-separated)",
        "player_name": "Player name to filter by",
        "min_lat": "Minimum latitude (default: 45470259)",
        "min_lng": "Minimum longitude (default: 12244155)",
        "max_lat": "Maximum latitude (default: 45480370)",
        "max_lng": "Maximum longitude (default: 12298207)",
        "min_timestamp": "Minimum timestamp (milliseconds or ISO 8601 format, default: -1)",
        "max_timestamp": "Maximum timestamp (milliseconds or ISO 8601 format, default: -1)"
      },
      "event_types": [
        "RESONATOR_DEPLOYED",
        "RESONATOR_DESTROYED",
        "PORTAL_CAPTURED",
        "PORTAL_NEUTRALIZED",
        "PORTAL_UNDER_ATTACK",
        "LINK_CREATED",
        "LINK_DESTROYED",
        "CONTROL_FIELD_CREATED",
        "UNKNOWN"
      ]
    }
  }
}

GET /plexts

Get plexts from the Ingress API with optional filters.

Query Parameters:

Parameter	Type	Default	Description
event_types	String	None	Comma-separated list of event types to filter by
player_name	String	None	Player name to filter by
min_lat	Integer	From env	Minimum latitude (E6 format)
min_lng	Integer	From env	Minimum longitude (E6 format)
max_lat	Integer	From env	Maximum latitude (E6 format)
max_lng	Integer	From env	Maximum longitude (E6 format)
min_timestamp	String	-1	Minimum timestamp (milliseconds or ISO 8601 format)
max_timestamp	String	-1	Maximum timestamp (milliseconds or ISO 8601 format)

Response:

{
  "count": 2,
  "plexts": [
    {
      "id": "abc123",
      "timestamp": 1736659207000,
      "timestamp_formatted": "2026-01-12 11:00:07",
      "text": "Albicocca captured L' Arboreto",
      "team": "RESISTANCE",
      "plext_type": "PLAYER_GENERATED",
      "categories": 0,
      "event_type": "PORTAL_CAPTURED",
      "player_name": "Albicocca",
      "portal_name": "L' Arboreto",
      "coordinates": {
        "lat": 45471652,
        "lng": 12274703
      },
      "markup": [
        {
          "type": "PLAYER",
          "plain": "Albicocca",
          "team": "RESISTANCE",
          "name": "Albicocca",
          "address": "",
          "latE6": 0,
          "lngE6": 0
        },
        {
          "type": "PORTAL",
          "plain": "L' Arboreto",
          "team": "RESISTANCE",
          "name": "L' Arboreto",
          "address": "Via Arboreto, Venice, Italy",
          "latE6": 45471652,
          "lngE6": 12274703
        }
      ]
    }
  ]
}

Examples:

# Get all plexts
curl "http://localhost:7000/plexts"

# Filter by event type
curl "http://localhost:7000/plexts?event_types=PORTAL_CAPTURED,LINK_CREATED"

# Filter by player
curl "http://localhost:7000/plexts?player_name=Albicocca"

# Filter by time range
curl "http://localhost:7000/plexts?min_timestamp=2026-01-12T10:00:00Z&max_timestamp=2026-01-12T12:00:00Z"

# Combine filters
curl "http://localhost:7000/plexts?event_types=PORTAL_CAPTURED&player_name=Albicocca&min_timestamp=2026-01-12T10:00:00Z"

Scheduler

The scheduler automatically collects Ingress events every minute and stores them in MongoDB.

Starting the Scheduler

# Start the scheduler
python schedule.py

# Or using the entrypoint script
./entrypoint-schedule.sh

How It Works

1. The scheduler runs every minute using APScheduler
2. It calculates a time range for the previous minute
3. Fetches plexts from the web API endpoint
4. Stores the results in MongoDB
5. Logs the number of plexts collected

Configuration

The scheduler requires the following environment variables:

• ENDPOINT_URL - The web API endpoint to fetch plexts from
• MONGO_URI - MongoDB connection string
• DB_NAME - Database name
• COLLECTION_NAME - Collection name
• TZ - Timezone for time calculations

Example Output

[2026-01-12 11:00:00 CET] Starting job...
Time range: 2026-01-12T10:59:00+01:00 to 2026-01-12T11:00:00+01:00
Fetched 5 plexts
Inserted 5 plexts to MongoDB
Job completed successfully

Docker Deployment

Using Docker Compose

The project includes a docker-compose.yml file for easy deployment.

Services:

1. MongoDB - Database service

   • Image: mongo:7.0
   • Port: 27017
   • Health checks enabled
   • Persistent volumes for data

2. Web - Flask web API

   • Image: mrosati84/ingress-web:1.0
   • Port: 7000
   • Depends on MongoDB
   • Health checks enabled

3. Schedule - Scheduler service

   • Image: mrosati84/ingress-schedule:1.0
   • Depends on MongoDB and Web
   • No exposed ports

Commands:

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f

# View logs for specific service
docker-compose logs -f web

# Stop all services
docker-compose down

# Stop and remove volumes
docker-compose down -v

# Restart a service
docker-compose restart web

# Scale services
docker-compose up -d --scale web=3

Building Custom Images

If you want to build your own Docker images:

# Build web image
docker build -f Dockerfile-web -t ingress-web .

# Build schedule image
docker build -f Dockerfile-schedule -t ingress-schedule .

# Update docker-compose.yml to use your images

Health Checks

All services include health checks:

• MongoDB: Database ping test
• Web: HTTP GET request to root endpoint
• Schedule: No health check (depends on other services)

Event Types

Event Type	Description
RESONATOR_DEPLOYED	Player deploys resonator on portal
RESONATOR_DESTROYED	Resonator destroyed
PORTAL_CAPTURED	Portal captured by player
PORTAL_NEUTRALIZED	Portal neutralized
PORTAL_UNDER_ATTACK	Portal being attacked
LINK_CREATED	Link created between portals
LINK_DESTROYED	Link destroyed
CONTROL_FIELD_CREATED	Control field created
UNKNOWN	Unrecognized event type

Output Format

CLI Output

Events are printed in the following format:

[2026-01-12 10:28:27] [PORTAL_CAPTURED] Albicocca captured L' Arboreto - Coords: 45471652, 12274703
[2026-01-12 10:28:26] [LINK_CREATED] Albicocca linked from L' Arboreto to Parco San Giuliano - Coords: 45471652, 12274703

API Output

The API returns JSON with the following structure:

{
  "count": 2,
  "plexts": [
    {
      "id": "string",
      "timestamp": 1736659207000,
      "timestamp_formatted": "2026-01-12 11:00:07",
      "text": "string",
      "team": "RESISTANCE",
      "plext_type": "PLAYER_GENERATED",
      "categories": 0,
      "event_type": "PORTAL_CAPTURED",
      "player_name": "string",
      "portal_name": "string",
      "coordinates": {
        "lat": 45471652,
        "lng": 12274703
      },
      "markup": [...]
    }
  ]
}

Project Structure

ingress/
├── main.py                    # CLI entry point
├── app.py                     # Flask web API
├── ingress.py                 # Ingress API client
├── models.py                  # Data models (Plext, EventType, Markup)
├── schedule.py                # Scheduled data collector
├── pyproject.toml             # Project configuration (uv)
├── requirements.txt           # Python dependencies
├── docker-compose.yml         # Docker orchestration
├── Dockerfile-web             # Web service Docker image
├── Dockerfile-schedule        # Scheduler Docker image
├── entrypoint-web.sh          # Web service entrypoint script
├── entrypoint-schedule.sh     # Scheduler entrypoint script
├── json_doc.md                # API documentation
├── .python-version            # Python version specification
├── .gitignore                 # Git ignore rules
├── .env.example               # Environment variables template
└── README.md                  # This file

Dependencies

Core Dependencies

• Python 3.11+ (Docker images use Python 3.11)
• requests>=2.31.0 - HTTP client for API requests
• flask>=3.1.2 - Web framework for REST API
• gunicorn>=23.0.0 - Production WSGI server
• python-dotenv>=1.2.1 - Environment variable management
• pymongo>=4.10.0 - MongoDB driver
• apscheduler>=3.10.0 - Task scheduling
• ipython>=9.9.0 - Interactive Python shell (development)

Development Dependencies

• uv - Fast Python package installer (optional)

Development

Local Development Setup

# Install dependencies
pip install -r requirements.txt

# Or using uv
uv sync

# Set up environment variables
cp .env.example .env
# Edit .env with your configuration

# Start MongoDB (if not using Docker)
# Install MongoDB locally or use a cloud instance

# Start the web API
python app.py

# In another terminal, start the scheduler
python schedule.py

# Test the CLI
python main.py --help

Running Tests

# Run CLI tests
python main.py --event-types PORTAL_CAPTURED --player-name test

# Test API endpoints
curl http://localhost:7000/
curl http://localhost:7000/plexts

Debugging

The application includes comprehensive logging:

• CLI: Logs to stdout
• Web API: Logs to stdout (Gunicorn captures and forwards)
• Scheduler: Logs to stdout
• Ingress API Client: Detailed DEBUG level logging

To enable debug logging:

# Set log level in environment
export LOG_LEVEL=DEBUG

# Or modify the logging configuration in the source files

Troubleshooting

Common Issues

Issue: "No CSRF token found in cookie!"

• Solution: Ensure your INGRESS_COOKIE includes both csrftoken and sessionid

Issue: "MongoDB connection failed"

• Solution: Check that MongoDB is running and MONGO_URI is correct

Issue: "Scheduler not collecting data"

• Solution: Verify ENDPOINT_URL is accessible and web API is running

Issue: "Docker containers not starting"

• Solution: Check Docker logs: docker-compose logs

Issue: "Health check failing"

• Solution: Ensure all dependencies are running and ports are accessible

Debug Mode

Enable debug mode for more detailed logging:

# For web API
export FLASK_DEBUG=1
python app.py

# For scheduler
export APSCHEDULER_DEBUG=1
python schedule.py

Logs

View logs for different components:

# Docker logs
docker-compose logs -f

# Specific service logs
docker-compose logs -f web
docker-compose logs -f schedule
docker-compose logs -f mongodb

# Application logs (if running locally)
# Logs are printed to stdout

License

See project license file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Support

For issues and questions, please open an issue on the project repository.