backfill

A Rails engine for queueing and managing data migrations.

Batched, checkpointed, resumable data operations for Laravel — backfills, cleanups, and migrations that survive deploys, crashes, and Ctrl+C.

Fix "Codex cannot access its local database" / "migration 1 was previously applied but has been modified" / "timed out waiting for state db backfill after 30s" on Codex Desktop (Windows/WSL/Mac/Linux) after the 0.130 → 0.131 update. CRLF/LF sqlx checksum drift in state_5.sqlite + logs_2.sqlite. Sessions preserved. See openai/codex#23787.

Ansible role to setup downsampling with continuous queries on influxDB, optionally with backfilling and compaction of existing raw data.

A plugin for backfilling task's and dag's through the UI

Backfill historical OHLC feature in a Feature Store (Hopsworks) using an orchestration tool (Prefect).

Tai Pan WebSocket RTD Client for AmiBroker – includes C# Feed Server and Python Bridge with INFO metadata integration.

zhao's dbt microbatch cascading time-window planner addon (AGPLv3)

Pyscript service to retroactively recompute Bayesian binary sensor state history in Home Assistant

A cross-browser extension (Chrome, Firefox, Safari) that collects your own Instagram post URLs with timestamps by stepping through your profile, then exports them as JSON.

Data backfill extension for WeaveVM-ExEx data protocol on Arweave

End-to-end Weather ETL built with Apache Airflow, MinIO, and PostgreSQL. Extract Open-Meteo hourly data → normalize to Parquet → load to L1/L2 tables with idempotent, backfillable DAGs, simple data checks, and SQL-only transformations.

Replays real Slurm job traces against multifactor priority and EASY backfill — a discrete-event simulator for testing scheduling policy before it reaches a production controller.

Convert C.M.I. CSV export into OpenMetrics text file format

Daily sync and weekly backfill that converge on one dedup-by-latest upsert, so re-running any range is safe. Idempotent backfill, demonstrated.

SQL playground using NYC Taxi Data. Dockerized PostgreSQL, pgAdmin, and Superset are included. Database initialization, backfill, and data ingestion processes are preconfigured and ready to go. Start analytics fast

Idempotent backfill orchestrator: chunked date-range backfills with a durable SQLite ledger, lease-based crash recovery proven by a SIGKILL test, collision leases against live loads, and an Airflow DAG factory. Overhead measured at 0.24-1.7ms per chunk (13.8% on a real 365-day load).

Building Data Pipelines for a data warehouse with Airflow and AWS

Standalone, partitioned, resumable Elasticsearch 5.6 -> OpenSearch 2.x backfill worker

Airflow Patterns - The orchestration patterns worth copying: idempotent backfills, dynamic task mapping, SLAs, and DAGs that are unit tested