diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/README.md b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/README.md index 19d09757af..9e87b9b3aa 100644 --- a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/README.md +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/README.md @@ -1,4 +1,3 @@ - # India_RBI_State_Statistics ## Import Overview @@ -10,30 +9,35 @@ This import pipeline processes various socio-economic and agricultural statistic ## Configuration -The `rbi_download.py` script relies on a configuration file named `configs.py` to fetch the URLs, filenames, and categories for the data to be downloaded. +The `rbi_download.py` script relies on a local, version-controlled JSON configuration file named `configs.json` to specify URLs, filenames, and categories for the data to be downloaded. -Sample data of the config file: -URLS_CONFIG=[ +Sample structure of `configs.json`: +```json +{ + "URLS_CONFIG": [ { - "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/58T_xxxxxxxxxxxxx.XLSX", - "category": "agriculture", - "filename": "state_wise_pattern_of_land_use_gross_sown_area.xlsx" - }, - ... - ] + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/58T_xxxxxxxxxxxxx.XLSX", + "category": "agriculture", + "filename": "state_wise_pattern_of_land_use_gross_sown_area.xlsx" + } + ] +} +``` -This approach makes the import process semi-automatic: if the download URLs change in future RBI releases (which commonly happens due to shifting table numbers or publication structures), only this configuration file needs to be updated, rather than modifying the Python script itself. +This makes the import process **Semi-Automated**: if download URLs change in future RBI releases (due to updated publication links or revised tables), only `configs.json` needs to be updated without modifying script logic. Additionally, `rbi_download.py` employs connection pooling with `requests.Session()` + `HTTPAdapter(Retry(...))`, validates file magic bytes (`PK\x03\x04`), and atomically writes downloaded files to prevent corrupted/partial files. ## Data Acquisition and Initial Preprocessing -The `rbi_download.py` script is responsible for both downloading the raw Excel data and performing an initial preprocessing step. This preprocessing involves reading all sheets from the downloaded `.xlsx` files and replacing any `*` symbols with empty values. +The `rbi_download.py` script is responsible for both downloading the raw Excel data and performing an initial preprocessing step: +- **Cell Cleaning (`clean_cell`)**: Strips footnote markers (`*` and `@`), trims leading/trailing whitespace, and converts empty or literal `'nan'` strings to true `NaN` values so `stat_var_processor.py` does not emit malformed string observations. +- **Header Normalization (`safe_to_numeric`)**: Converts numeric or float-coerced year headers (e.g., `2015.0`, `'2016@'`) in `State/Union Territory` header rows to integers (`2015`, `2016`) so they match integer string keys in the PVMaps. ### How to Run: Execute the `rbi_download.py` script. This script will: 1. Automatically create an `input_files` directory if it doesn't exist. -2. Download the necessary Excel files into subfolders within `input_files` (e.g., `input_files/agriculture/`). -3. Process each downloaded Excel file by reading all its sheets and replacing `*` characters with empty strings in all cells. +2. Download the necessary Excel files into subfolders within `input_files` (e.g., `input_files/agriculture/`), verifying `PK\x03\x04` ZIP signatures and writing files atomically. +3. Process each downloaded Excel file across all sheets using `clean_cell` and `safe_to_numeric`. ### Download Command: @@ -41,20 +45,54 @@ Execute the `rbi_download.py` script. This script will: python3 rbi_download.py ``` +Optional flag: +- `--config_file_path`: Path to the JSON config file (defaults to local `configs.json`; also supports a GCS URI such as `gs:///configs.json`). + +### Running Unit Tests: + +To run the hermetic unit test suite for `rbi_download.py`: + +```bash +python3 rbi_download_test.py +``` + +## Prerequisites + +This import relies on dependencies managed in the repository virtual environment (`.env`). + +To initialize and activate the repository virtual environment from the repository root (`/data/`): +```bash +./run_tests.sh -r +source .env/bin/activate +``` + +Alternatively, invoke Python directly using the virtual environment interpreter: +```bash +.env/bin/python rbi_download.py +``` + +The required packages managed in `requirements_all.txt` include: +- `pandas` +- `openpyxl` +- `requests` +- `google-cloud-storage` +- `absl-py` + ## Processing Section -The downloaded data is processed using the `stat_var_processor.py` script, which is part of the `/data/tools/statvar_importer/` toolkit. This script converts the raw Excel data into a structured format suitable for further analysis and ingestion. Each processing command specifies the input data file(s), the Property-Value (PV) map, the configuration file (metadata), a places resolver CSV, and the desired output path. +The downloaded data is processed using the `stat_var_processor.py` script, which is part of the `/data/tools/statvar_importer/` toolkit. This script converts the raw Excel data into a structured format suitable for further analysis and ingestion. Each processing command specifies the input data file(s), the Property-Value (PV) map, the configuration file (metadata), a places resolver CSV, counter output CSV, and the desired output path. ### General Processing Command Structure: ```bash python3 stat_var_processor.py \ - --input_data='.xlsx' \ - --pv_map='' \ - --config_file='' \ - --places_resolved_csv='' \ - --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ - --output_path='' + --input_data=".xlsx" \ + --pv_map="" \ + --config_file="" \ + --places_resolved_csv="" \ + --existing_statvar_mcf="gs://unresolved_mcf/scripts/statvar/stat_vars.mcf" \ + --output_counters="" \ + --output_path="" ``` ### How to Run Processing: @@ -69,7 +107,7 @@ sh run.sh #### Option 2: Executing Individual Processing Commands -Navigate to the `/data/tools/statvar_importer/` directory before running the following commands, or adjust the relative paths accordingly. +Navigate to the `/data/tools/statvar_importer/` directory before running the following commands, or adjust the relative paths accordingly. Note that wildcard `--input_data` paths must be enclosed in double quotes so `stat_var_processor.py` expands them internally. ### Processing Commands: @@ -77,11 +115,12 @@ Navigate to the `/data/tools/statvar_importer/` directory before running the fol ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/agriculture/*.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/agriculture/*.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/agriculture_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/agriculture_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/agriculture/agriculture_output ``` @@ -89,11 +128,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/state_wise_forest_cover.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/state_wise_forest_cover.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_forest_cover_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/environment/state_wise_forest_cover_output ``` @@ -101,11 +141,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/state_wise_tree_cover.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/state_wise_tree_cover.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_tree_cover_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/environment/state_wise_tree_cover_output ``` @@ -113,11 +154,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/sub_division_wise_annual_rainfall.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/sub_division_wise_annual_rainfall.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/sub_division_wise_annual_rainfall_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/environment/sub_division_wise_annual_rainfall_output ``` @@ -125,11 +167,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/state_wise_expenditure_on_relief_on_natural_calamities.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/state_wise_expenditure_on_relief_on_natural_calamities.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_expenditure_on_relief_on_natural_calamities_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/environment/state_wise_expenditure_on_relief_on_natural_calamities_output ``` @@ -137,11 +180,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/state_wise_sustainable_development_goals_score_SDGs.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/environment/state_wise_sustainable_development_goals_score_SDGs.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_sdg_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_sdg_score_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/environment/state_wise_sdg_score_output ``` @@ -149,11 +193,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/price_and_wages/*.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/price_and_wages/*.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/price_wages_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/price_and_wages_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/price_and_wages/price_and_wages_output ``` @@ -161,11 +206,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_per_capita_availability_of_power.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_per_capita_availability_of_power.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_per_capita_availability_of_power_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_per_capita_availability_of_power_output ``` @@ -173,11 +219,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_availability_of_power.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_availability_of_power.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_availability_of_power_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_availability_of_power_output ``` @@ -185,11 +232,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_installed_capacity_of_power.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_installed_capacity_of_power.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_installed_capacity_of_power_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_installed_capacity_of_power_output ``` @@ -197,11 +245,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_power_requirement.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_power_requirement.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_power_requirement_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_power_requirement_output ``` @@ -209,11 +258,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_length_of_national_highways.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_length_of_national_highways.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_length_of_national_highways_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_length_of_national_highways_output ``` @@ -221,11 +271,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_railway_route.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_railway_route.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_railway_route_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_railway_route_output ``` @@ -233,11 +284,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_length_of_roads.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_length_of_roads.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_length_of_roads_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_length_of_roads_output ``` @@ -245,11 +297,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_length_of_state_highways.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_length_of_state_highways.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_length_of_state_highways_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_length_of_state_highways_output ``` @@ -257,11 +310,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_electricity_transmission_distribution_losses.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_electricity_transmission_distribution_losses.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_electricity_transmission_distribution_losses_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_electricity_transmission_distribution_losses_output ``` @@ -269,11 +323,12 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_telephones_per_100_population.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_telephones_per_100_population.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_telephones_per_100_population_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_telephones_per_100_population_output ``` @@ -281,10 +336,75 @@ python3 stat_var_processor.py \ ```bash python3 stat_var_processor.py \ - --input_data=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_road_constructed_under_PMGSY.xlsx \ + --input_data="../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/input_files/infrastructure/state_wise_road_constructed_under_PMGSY.xlsx" \ --pv_map=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/infrastructure_pvmap.csv \ --config_file=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_metadata.csv \ --places_resolved_csv=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv \ --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf \ + --output_counters=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/counters/state_wise_road_constructed_under_pmgsy_counters.csv \ --output_path=../../statvar_imports/database_on_indian_economy/india_rbi_state_statistics/output_files/infrastructure/state_wise_road_constructed_under_pmgsy_output ``` + +## Validation Configuration and Thresholds + +This import uses `validation_config.json` with: +1. **`DELETED_RECORDS_PERCENT: 0.1` (`check_deleted_records_percent`)**: + - **Rationale & Ingestion Strategy**: The canonical threshold is strictly set to `0.1%` + (`threshold: 0.1`), adhering to the Master CRA standard ceiling (threshold <= 0.5%). + During the initial baseline rollout, one-time deletions occur across 8 sub-imports + (**100%** in `state_wise_sdg_score_output.csv`, **6.94%** in `sub_division_wise_annual_rainfall_output.csv`, **5.64%** in + `agriculture_output.csv`, **3.15%** in + `state_wise_expenditure_on_relief_on_natural_calamities_output.csv`, **1.39%** in + `state_wise_installed_capacity_of_power_output.csv`, **0.28%** in + `state_wise_tree_cover_output.csv`, **0.16%** in `state_wise_forest_cover_output.csv`, + and **0.15%** in `state_wise_electricity_transmission_distribution_losses_output.csv`). + These one-time deletions are coordinated via a one-time baseline reset / staging + ingestion override with tech leads to promote the new output to `latest_version`. + All subsequent automated runs will strictly adhere to the `0.1%` threshold against + the updated baseline. Root causes for the one-time deletions are: + 1. **StatVar Schema Alignment (Table 106)**: Renaming `Score_SustainableDevelopmentGoals` + to `IndexScore_SustainableDevelopmentGoal` (`measuredProperty: indexScore`, + `populationType: SustainableDevelopmentGoal`) in `environment_pvmap.csv` (149 records / + 100% in `state_wise_sdg_score_output.csv`, replaced by 184 records under the canonical StatVar). + 2. **Place Resolution Corrections**: Remapping `"Haryana, Delhi & Chandigarh"` from + `dcid:wikidataId/Q1174` (Haryana) to the dedicated IMD sub-division DCID + `dcid:HaryanaDelhiChandigarh` (24 records / 6.94% in rainfall), and fixing + `NCT of Delhi` from `wikidataId/Q1352` (erroneously Chennai) to `wikidataId/Q1353` + (resolving all Delhi records to canonical `wikidataId/Q1353`). + 3. **Unit Schema Corrections**: Changing `Annual_Amount_FarmInventory_Fruits` in + `agriculture_pvmap.csv` from `Hectare` to `MetricTon` (309 records in agriculture, + where Import Differ classifies old unit nodes as deleted and new unit nodes as + added). + 4. **Official RBI Source Revisions**: Retrospective updates by RBI to provisional + figures for recent fiscal years (`2022-03` through `2024-03`) and UT consolidations + (`Dadra & Nagar Haveli and Daman & Diu`). + - **Technical Breakdown of Deletions**: + | Category | Output File | Deletion % / Records | Root Cause | + |---|---|---|---| + | Environment | `state_wise_sdg_score_output.csv` | 100.0% (149 records) | Renamed `Score_SustainableDevelopmentGoals` to canonical `IndexScore_SustainableDevelopmentGoal` (184 new observations emitted). | + | Environment | `sub_division_wise_annual_rainfall_output.csv` | 6.94% (24 records) | Remapped `"Haryana, Delhi & Chandigarh"` from `dcid:wikidataId/Q1174` (Haryana state) to dedicated IMD sub-division `dcid:HaryanaDelhiChandigarh`. | + | Agriculture | `agriculture_output.csv` | 5.64% (1,282 records) | Corrected `NCT of Delhi` to `wikidataId/Q1353` (439 records); updated `Annual_Amount_FarmInventory_Fruits` unit to `MetricTon` (309 records); retrospective RBI revisions for provisional years `2022-03`–`2024-03` (534 records). | + | Environment | `state_wise_expenditure_on_relief_on_natural_calamities_output.csv` | 3.15% (13 records) | Corrected `NCT of Delhi` place resolution from `wikidataId/Q1352` (Chennai) to `wikidataId/Q1353`. | + | Infrastructure | `state_wise_installed_capacity_of_power_output.csv` | 1.39% (10 records) | `country/IND` (2015–2024) dropped because RBI sheet T_140(ii) omits ALL INDIA totals. | + | Environment | `state_wise_tree_cover_output.csv` | 0.28% (1 record) | Daman & Diu (`wikidataId/Q66710`, 2013) UT merger revision. | + | Environment | `state_wise_forest_cover_output.csv` | 0.16% (1 record) | Daman & Diu (`wikidataId/Q66710`, 2013) UT merger revision. | + | Infrastructure | `state_wise_electricity_transmission_distribution_losses_output.csv` | 0.15% (1 record) | Dadra & Nagar Haveli (`wikidataId/Q46107`, 2023-03) UT merger revision. | +2. **`SQL_VALIDATOR` non-empty check (`check_expected_statvar_count`)**: + - Asserts `statvar_cnt >= 1` so an empty `summary_report.csv` cannot vacuously pass grouped SQL queries. +3. **`SQL_VALIDATOR` cadence freshness check (`check_max_date_freshness`)**: + - Asserts that each StatVar meets its expected RBI publication cadence (`2020`, `2022`, `2023`, or `2024`) with `WHERE MaxDate IS NOT NULL`. + +## Refresh & Operational Maintenance + +The import is configured with an automated cron schedule in `manifest.json`: +- **Schedule:** `0 10 * * 1` (Every Monday at 10:00 AM UTC). +- **Import Type:** Semi-Automated. + +### Annual Refresh Procedure: +1. When the Reserve Bank of India publishes a new edition of the *Handbook of Statistics on Indian States*, check the [RBI Annual Publications Portal](https://www.rbi.org.in/Scripts/AnnualPublications.aspx?head=Handbook%20of%20Statistics%20on%20Indian%20States). +2. If table download URLs have changed, update the `url` fields in `configs.json`. +3. If new tables, columns, or indicators are introduced, update the corresponding PVMap (`agriculture_pvmap.csv`, `environment_pvmap.csv`, `infrastructure_pvmap.csv`, or `price_wages_pvmap.csv`). +4. Re-run `python3 rbi_download.py` followed by `sh run.sh` to download and process the updated datasets. +5. Verify test outputs and pre-submit validation before raising a pull request. + + diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/agriculture_pvmap.csv b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/agriculture_pvmap.csv index 1110d3b0e4..fff34a4a85 100644 --- a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/agriculture_pvmap.csv +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/agriculture_pvmap.csv @@ -81,4 +81,4 @@ TABLE 94: STATE-WISE WOOL PRODUCTION,populationType,FarmInventory,measuredProper 2029-30,value,{Number},observationDate,2030-03,observationPeriod,P1Y,,,,,, 2030-31,value,{Number},observationDate,2031-03,observationPeriod,P1Y,,,,,, 2031-32,value,{Number},observationDate,2032-03,observationPeriod,P1Y,,,,,, -2032-33,value,{Number},observationDate,2033-03,observationPeriod,P1Y,,,,,, \ No newline at end of file +2032-33,value,{Number},observationDate,2033-03,observationPeriod,P1Y,,,,,, diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/configs.json b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/configs.json new file mode 100644 index 0000000000..023961b31b --- /dev/null +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/configs.json @@ -0,0 +1,334 @@ +{ + "URLS_CONFIG": [ + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/53T_111220253720381F6FEB46769E9A3BA1775126F7.XLSX", + "category": "agriculture", + "filename": "state_wise_pattern_of_land_use_gross_sown_area.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/54T_111220257E02BB1F59FC46F2B936CC773BBC0874.XLSX", + "category": "agriculture", + "filename": "state_wise_pattern_of_land_use_net_sown_area.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/55T_11122025AFBC97CC9BA44E54BC31B3E58DBF2E21.XLSX", + "category": "agriculture", + "filename": "state_wise_pattern_of_land_use_gross_irrigated_area.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/56T_1112202564A24EAD0AA44A278DB069916CA8FECB.XLSX", + "category": "agriculture", + "filename": "state_wise_pattern_of_land_use_net_irrigated_area.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/57T_111220253F1BCFDCDF5047B495821F992E4D87B6.XLSX", + "category": "agriculture", + "filename": "state_wise_pattern_of_land_use_cropping_intensity.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/58T_111220250E85BAD71CD64324A6F4E28DD0A05129.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_foodgrains_rice.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/59T_1112202571AAC5A686F74224A1DF55D2B9D62D77.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_foodgrains_wheat.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/60T_111220251CE8EB785C6E40FAA8BA4EE805D01B93.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_foodgrains_coarse_cereals.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/61T_111220251C3BA74BCC3742A7BA09518C4DDD0BE5.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_foodgrains_pulses.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/62T_11122025F9D0951B74274E99A9C236A5CFAC1858.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_total_foodgrains.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/63T_1112202514B06350BA2849F98C3551060541C568.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_non_foodgrains_oilseeds.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/64T_111220251AE568D097A740F18A52EFEB30BDDA02.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_non_foodgrains_cotton_lintt.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/65T_11122025FF1A1A01BD604A1A9A63E572FF11B5B5.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_non_foodgrains_sugarcane.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/66T_1112202551DDDB3EE6624BE9B30EB1692CC0CBA6.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_non_foodgrains_raw_jute_mesta.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/67T_1112202594892EEE85FF4BAC9AAB5DC35661E62A.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_foodgrains_rice.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/68T_11122025D202C9C812754B4FBC0F25A348940AA8.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_foodgrains_wheat.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/69T_111220258B2ACEEE047E482AA3EDF0C09F7E3A86.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_foodgrains_coarse_cereals.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/70T_11122025A43EF0C6E0154357B8E404D9CF42998C.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_foodgrains_pulses.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/71T_111220252D54659F49874E689923BF46C29E01A5.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_total_foodgrains.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/72T_11122025D573A69D2317423FBCFB65F5622AF929.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_total_fruits.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/73T_11122025E35C4283EAE7430A88B1BF32B236A976.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_total_vegetables.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/74T_11122025EBF01225C48441C2A23B436CCC589BF1.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_total_fruits.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/75T_11122025C8A307B91418448DBF5B5E54997E7355.XLSX", + "category": "agriculture", + "filename": "state_wise_production_of_total_vegetables.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/76T_1112202535D900531AEA4C71B96F562B9AE59835.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_non_foodgrains_oilseeds.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/77T_11122025DA3881A5159A4760B53AC35C5BFBA8D9.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_non_foodgrains_cotton_lintt.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/78T_111220252516A0D23AA94A6B809CFB3B1953DC7B.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_non_foodgrains_sugarcane.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/79T_1112202567B7934A72A04BF988AAC0656AA9DB66.XLSX", + "category": "agriculture", + "filename": "state_wise_area_of_non_foodgrains_raw_jute_mesta.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/80T_111220251421B93FB22A4B8BAE3E52809636B603.XLSX", + "category": "agriculture", + "filename": "state_wise_estimates_of_yield_rice.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/81T_11122025F795A58AD4964BFD9C1AF74F0557164E.XLSX", + "category": "agriculture", + "filename": "state_wise_estimates_of_yield_wheat.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/82T_1112202534205116BBCA4010AC1792E31FADC9C9.XLSX", + "category": "agriculture", + "filename": "state_wise_estimates_of_yield_coarse_cereals.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/83T_111220255A0477BFBDFD45AF93EDA200B2A5A66F.XLSX", + "category": "agriculture", + "filename": "state_wise_estimates_of_yield_pulses.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/84T_11122025D106F74C0F98465DAF95013CB77F3A20.XLSX", + "category": "agriculture", + "filename": "state_wise_estimates_of_yield_total_foodgrains.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/85T_111220250039FAA1932944E5B7627528B147FAFC.XLSX", + "category": "agriculture", + "filename": "state_wise_estimates_of_yield_oilseeds.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/86T_111220258B6992228C504C2D84CBD88574260E87.XLSX", + "category": "agriculture", + "filename": "state_wise_estimates_of_yield_cotton_lintt.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/87T_1112202547D1DA4285204EAFA0BAAA6CA47834C7.XLSX", + "category": "agriculture", + "filename": "state_wise_estimates_of_yield_sugarcane.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/88T_1112202507582D4179654DC292A6311E3BDB2AB5.XLSX", + "category": "agriculture", + "filename": "state_wise_estimates_of_yield_raw_jute_mesta.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/89T_11122025CC34A78CE2D64C1D880A9B4B6FA1135D.XLSX", + "category": "agriculture", + "filename": "state_wise_per_hectare_consumption_of_fertiliser_n_p_k.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/90T_1112202558892C39CD554E4098A8EB79C63D47D9.XLSX", + "category": "agriculture", + "filename": "state_wise_meat_production.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/91T_11122025A7A12D69EA1645129226BEAE5783E1A4.XLSX", + "category": "agriculture", + "filename": "state_wise_milk_production.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/92T_11122025867917E0AC5644ADB035966AAF9D220B.XLSX", + "category": "agriculture", + "filename": "state_wise_egg_production.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/93T_1112202561B6558711FE4088998165985924266E.XLSX", + "category": "agriculture", + "filename": "state_wise_fish_production.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/94T_111220254142688D61EB497BA75DAFDDB72F7160.XLSX", + "category": "agriculture", + "filename": "state_wise_wool_production.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/99T_111220251AFF6E04A39B48E499CE9CDE9E66838B.XLSX", + "category": "environment", + "filename": "state_wise_forest_cover.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/101T_11122025C61D6934B6D74891B15B91679CFE1689.XLSX", + "category": "environment", + "filename": "state_wise_tree_cover.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/102T_11122025C0C87B2909234302A1F4A52A9A42E4DF.XLSX", + "category": "environment", + "filename": "sub_division_wise_annual_rainfall.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/105T_11122025454345A0E067418D9326DD30AE67BF6F.XLSX", + "category": "environment", + "filename": "state_wise_expenditure_on_relief_on_natural_calamities.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/106T_111220251B177E8F85CA412DB39DCE90BCBD724A.XLSX", + "category": "environment", + "filename": "state_wise_sustainable_development_goals_score_SDGs.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/138T_111220252D21BC51E5A842E4B25E954391C10A41.XLSX", + "category": "infrastructure", + "filename": "state_wise_per_capita_availability_of_power.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/139T_111220256569FF1C11E248B7A8ED64D31C999776.XLSX", + "category": "infrastructure", + "filename": "state_wise_availability_of_power.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/140T_111220254D8DA0B69B444492B6E9BAF30F3395C8.XLSX", + "category": "infrastructure", + "filename": "state_wise_installed_capacity_of_power.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/141T_11122025615E98A72EA6475CAACFFD12FFC83057.XLSX", + "category": "infrastructure", + "filename": "state_wise_power_requirement.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/144T_111220250497208D070C44219818788DB5BCE35A.XLSX", + "category": "infrastructure", + "filename": "state_wise_length_of_national_highways.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/145T_111220259E033C52098E4732A6101C940DEF1222.XLSX", + "category": "infrastructure", + "filename": "state_wise_railway_route.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/146T_11122025B89449F4159B420E8305A9509F27FE82.XLSX", + "category": "infrastructure", + "filename": "state_wise_length_of_roads.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/147T_111220255C153B31AB2E4835ABEEAC91D7494BFA.XLSX", + "category": "infrastructure", + "filename": "state_wise_length_of_state_highways.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/148T_11122025D741C3CB2CC24F448DC99D907589DD71.XLSX", + "category": "infrastructure", + "filename": "state_wise_electricity_transmission_distribution_losses.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/149T_1112202538D6A609C55B4FEFBBC44BB6C5ED20F7.XLSX", + "category": "infrastructure", + "filename": "state_wise_telephones_per_100_population.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/150T_11122025C30EFE7568EA472C8AB0290288B160C0.XLSX", + "category": "infrastructure", + "filename": "state_wise_road_constructed_under_PMGSY.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/108T_111220251942D16B3BED4D73BE851D15D2329168.XLSX", + "category": "price_and_wages", + "filename": "state_wise_average_inflation_CPI_general.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/109T_111220250E067E49350B48659E5A50C3D357BB46.XLSX", + "category": "price_and_wages", + "filename": "state_wise_average_inflation_CPI_food_and_beverages.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/110T_11122025784C1BDC2482496B9E25DA1BA4B38A3D.XLSX", + "category": "price_and_wages", + "filename": "state_wise_average_inflation_CPI_fuel_and_light.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/111T_111220252F7D9704AB664C35A58770BAC73518DE.XLSX", + "category": "price_and_wages", + "filename": "state_wise_average_inflation_CPI_housing_urban.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/112T_11122025771E55EB688044CE8A85E52A901B5D22.XLSX", + "category": "price_and_wages", + "filename": "state_wise_average_daily_wage_rates_in_rural_india_men_construction_workers.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/113T_111220256FEBCC253D4A47419892925F897E2537.XLSX", + "category": "price_and_wages", + "filename": "state_wise_average_daily_wage_rates_in_rural_india_men_general_agricultural_labourers.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/114T_111220259CA566982E7F4BF38B3FDAC0D7587F2E.XLSX", + "category": "price_and_wages", + "filename": "state_wise_Average_daily_wage_rates_in_rural_india_men_horticulture_workers.xlsx" + }, + { + "url": "https://rbidocs.rbi.org.in/rdocs/Publications/DOCs/115T_1112202585CF99F21E654CED8990A734A043B3DA.XLSX", + "category": "price_and_wages", + "filename": "state_wise_average_daily_wage_rates_in_rural_india_men_non_agricultural_labourers.xlsx" + } + ] +} diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_pvmap.csv b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_pvmap.csv index 0a1f8a72a5..ccc09e9b2c 100644 --- a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_pvmap.csv +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_pvmap.csv @@ -3,9 +3,11 @@ TABLE 99: STATE-WISE FOREST COVER,measuredProperty,area,populationType,LandCover TABLE 101: STATE-WISE TREE COVER,measuredProperty,area,populationType,LandCover,landCoverType,Tree,name,"""Area of Land Cover: Tree""",unit,SquareKilometer,,,, TABLE 102: SUB-DIVISION-WISE ANNUAL RAINFALL,measuredProperty,rainfall,populationType,Place,measurementQualifier,Annual,name,"""Annual Rainfall""",unit,Millimeter,,,, Table 105: STATE-WISE EXPENDITURE ON RELIEF ON NATURAL CALAMITIES,measuredProperty,amount,populationType,EconomicActivity,activitySource,ExpenditureActivity,expenditureType,ReliefOnNaturalCalamity,name,"""Expenditure on relief on natural calamity""",#Multiply,100000,unit,INR -Table 106: STATE-WISE SUSTAINABLE DEVELOPMENT GOALS (SDGs) SCORE,measuredProperty,score,populationType,SustainableDevelopmentGoals,name,"""Sustainable Development Goals Score""",,,,,,,, +Table 106: STATE-WISE SUSTAINABLE DEVELOPMENT GOALS (SDGs) SCORE,measuredProperty,indexScore,populationType,SustainableDevelopmentGoal,name,"""Sustainable Development Goal Index Score""",,,,,,,, State/ Union Territory,observationAbout,{Data},,,,,,,,,,,, State/Union Territory,observationAbout,{Data},,,,,,,,,,,, +"Haryana, Delhi & Chandigarh",observationAbout,dcid:HaryanaDelhiChandigarh +"Haryana, Delhi and Chandigarh",observationAbout,dcid:HaryanaDelhiChandigarh 1987,observationDate,1987,value,{Number},,,,,,,,,, 1989,observationDate,1989,value,{Number},,,,,,,,,, 1991,observationDate,1991,value,{Number},,,,,,,,,, @@ -80,4 +82,3 @@ State/Union Territory,observationAbout,{Data},,,,,,,,,,,, 2030-31 (BE),observationDate,2031-03,value,{Number},observationPeriod,P1Y,,,,,,,, 2031-32 (BE),observationDate,2032-03,value,{Number},observationPeriod,P1Y,,,,,,,, 2032-33 (BE),observationDate,2033-03,value,{Number},observationPeriod,P1Y,,,,,,,, - diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_sdg_metadata.csv b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_sdg_metadata.csv index f11ac0b3c4..e9bfc28acd 100644 --- a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_sdg_metadata.csv +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/environment_sdg_metadata.csv @@ -1,4 +1,5 @@ parameter,value header_rows,3 -output_columns,"observationAbout,observationDate,variableMeasured,value,unit,observationPeriod" +output_columns,"observationAbout,observationDate,variableMeasured,value" mapped_rows,3 +generate_schema_mcf,False diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/manifest.json b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/manifest.json index efeca24ba2..5d3a7dd7f1 100644 --- a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/manifest.json +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/manifest.json @@ -15,83 +15,110 @@ "input_files/agriculture/*.xlsx", "input_files/environment/*.xlsx", "input_files/infrastructure/*.xlsx", - "input_files/price_and_wages/*.xlsx" + "input_files/price_and_wages/*.xlsx", + "counters/*.csv", + "configs.json", + "validation_config.json" ], "import_inputs": [ { "template_mcf": "output_files/agriculture/agriculture_output.tmcf", - "cleaned_csv": "output_files/agriculture/agriculture_output.csv" + "cleaned_csv": "output_files/agriculture/agriculture_output.csv", + "node_mcf": "output_files/agriculture/agriculture_output*.mcf" }, { "template_mcf": "output_files/environment/state_wise_expenditure_on_relief_on_natural_calamities_output.tmcf", - "cleaned_csv": "output_files/environment/state_wise_expenditure_on_relief_on_natural_calamities_output.csv" + "cleaned_csv": "output_files/environment/state_wise_expenditure_on_relief_on_natural_calamities_output.csv", + "node_mcf": "output_files/environment/state_wise_expenditure_on_relief_on_natural_calamities_output*.mcf" }, { "template_mcf": "output_files/environment/state_wise_forest_cover_output.tmcf", - "cleaned_csv": "output_files/environment/state_wise_forest_cover_output.csv" + "cleaned_csv": "output_files/environment/state_wise_forest_cover_output.csv", + "node_mcf": "output_files/environment/state_wise_forest_cover_output*.mcf" }, { "template_mcf": "output_files/environment/state_wise_sdg_score_output.tmcf", - "cleaned_csv": "output_files/environment/state_wise_sdg_score_output.csv" + "cleaned_csv": "output_files/environment/state_wise_sdg_score_output.csv", + "node_mcf": "output_files/environment/state_wise_sdg_score_output*.mcf" }, { "template_mcf": "output_files/environment/state_wise_tree_cover_output.tmcf", - "cleaned_csv": "output_files/environment/state_wise_tree_cover_output.csv" + "cleaned_csv": "output_files/environment/state_wise_tree_cover_output.csv", + "node_mcf": "output_files/environment/state_wise_tree_cover_output*.mcf" }, { "template_mcf": "output_files/environment/sub_division_wise_annual_rainfall_output.tmcf", - "cleaned_csv": "output_files/environment/sub_division_wise_annual_rainfall_output.csv" + "cleaned_csv": "output_files/environment/sub_division_wise_annual_rainfall_output.csv", + "node_mcf": "output_files/environment/sub_division_wise_annual_rainfall_output*.mcf" }, { "template_mcf": "output_files/price_and_wages/price_and_wages_output.tmcf", - "cleaned_csv": "output_files/price_and_wages/price_and_wages_output.csv" + "cleaned_csv": "output_files/price_and_wages/price_and_wages_output.csv", + "node_mcf": "output_files/price_and_wages/price_and_wages_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_availability_of_power_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_availability_of_power_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_availability_of_power_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_availability_of_power_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_electricity_transmission_distribution_losses_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_electricity_transmission_distribution_losses_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_electricity_transmission_distribution_losses_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_electricity_transmission_distribution_losses_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_installed_capacity_of_power_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_installed_capacity_of_power_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_installed_capacity_of_power_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_installed_capacity_of_power_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_length_of_national_highways_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_length_of_national_highways_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_length_of_national_highways_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_length_of_national_highways_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_length_of_roads_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_length_of_roads_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_length_of_roads_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_length_of_roads_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_length_of_state_highways_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_length_of_state_highways_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_length_of_state_highways_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_length_of_state_highways_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_per_capita_availability_of_power_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_per_capita_availability_of_power_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_per_capita_availability_of_power_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_per_capita_availability_of_power_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_power_requirement_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_power_requirement_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_power_requirement_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_power_requirement_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_railway_route_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_railway_route_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_railway_route_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_railway_route_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_road_constructed_under_pmgsy_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_road_constructed_under_pmgsy_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_road_constructed_under_pmgsy_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_road_constructed_under_pmgsy_output*.mcf" }, { "template_mcf": "output_files/infrastructure/state_wise_telephones_per_100_population_output.tmcf", - "cleaned_csv": "output_files/infrastructure/state_wise_telephones_per_100_population_output.csv" + "cleaned_csv": "output_files/infrastructure/state_wise_telephones_per_100_population_output.csv", + "node_mcf": "output_files/infrastructure/state_wise_telephones_per_100_population_output*.mcf" } ], - "cron_schedule": "0 10 * * 1" + "resource_limits": { + "cpu": 1, + "memory": 2, + "disk": 30 + }, + "cron_schedule": "0 10 * * 1", + "validation_config_file": "validation_config.json" } ] -} \ No newline at end of file +} diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_download.py b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_download.py index e30edc2db1..715335ba78 100644 --- a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_download.py +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_download.py @@ -12,111 +12,382 @@ # See the License for the specific language governing permissions and # limitations under the License. +import json import os -from absl import app, logging, flags -import sys +import re +import tempfile +import time +from absl import app, flags, logging from google.cloud import storage import pandas as pd +import requests +from requests.adapters import HTTPAdapter +from urllib3.util import Retry SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__)) -sys.path.append(os.path.join(SCRIPT_DIR, '../../../util')) INPUT_DIR = os.path.join(SCRIPT_DIR, "input_files") +DEFAULT_CONFIG_JSON = os.path.join(SCRIPT_DIR, 'configs.json') -from download_util_script import _retry_method +flags.DEFINE_string('config_file_path', DEFAULT_CONFIG_JSON, + 'Config file path (local json or gs:// path)') +flags.DEFINE_bool('force_download', False, + 'If true, redownload files even if valid local copies exist.') -flags.DEFINE_string( - 'config_file_path', - 'gs://unresolved_mcf/india_rbi/state_statistics/configs.py', - 'Config file path') +DEFAULT_HEADERS = { + 'User-Agent': ('Mozilla/5.0 (Windows NT 10.0; Win64; x64) ' + 'AppleWebKit/537.36 (KHTML, like Gecko) ' + 'Chrome/124.0.0.0 Safari/537.36'), + 'Accept': + 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8', + 'Accept-Language': + 'en-US,en;q=0.9', + 'Referer': + 'https://www.rbi.org.in/', +} -def reads_config_file(): - _FLAGS = flags.FLAGS - config_file_path = _FLAGS.config_file_path +RBI_PORTAL_URL = ( + 'https://www.rbi.org.in/Scripts/AnnualPublications.aspx?' + 'head=Handbook%20of%20Statistics%20on%20Indian%20States' +) +XLSX_ZIP_SIGNATURE = b'PK\x03\x04' + + +def create_retry_session( + retries: int = 4, + backoff_factor: float = 2.0, + status_forcelist: tuple = (429, 500, 502, 503, 504), +) -> requests.Session: + """Creates a requests.Session with connection pooling, retries, and browser headers.""" + session = requests.Session() + retry_strategy = Retry( + total=retries, + backoff_factor=backoff_factor, + status_forcelist=status_forcelist, + allowed_methods=["HEAD", "GET"], + raise_on_status=False, + ) + adapter = HTTPAdapter(max_retries=retry_strategy, + pool_connections=10, + pool_maxsize=10) + session.mount("https://", adapter) + session.mount("http://", adapter) + session.headers.update(DEFAULT_HEADERS) + session.verify = True + return session + + +def _load_local_config(file_path): + if not os.path.exists(file_path): + return None try: - storage_client = storage.Client() - bucket_name = config_file_path.split('/')[2] - bucket = storage_client.bucket(bucket_name) - blob_name = '/'.join(config_file_path.split('/')[3:]) - blob = bucket.blob(blob_name) - file_contents = blob.download_as_text() - local_vars = {} - exec(file_contents, {}, local_vars) - return local_vars + with open(file_path, 'r', encoding='utf-8') as f: + return json.load(f) except Exception as e: - logging.fatal(f"Cannot extract url and related configs: {e}") + logging.warning(f"Failed to read local config file '{file_path}': {e}") + return None + -def download_files(URL_CONFIG): +def reads_config_file(): + _FLAGS = flags.FLAGS try: - for config in URL_CONFIG: - config_url = config.get("url") - category_name = config.get("category") - file_name = config.get("filename") - file_path = os.path.join(INPUT_DIR, category_name, file_name) - os.makedirs(os.path.join(INPUT_DIR, category_name), exist_ok=True) - file_response = _retry_method(config_url, headers=None, tries=3, delay=5, backoff=2) - if file_response: - with open(file_path, 'wb') as f: - f.write(file_response.content) - logging.info(f"Downloaded the file {file_name} successfully.") - except Exception as e: - logging.fatal(f"Download error: {str(e)}") + config_file_path = _FLAGS.config_file_path + except (flags.UnparsedFlagAccessError, AttributeError): + try: + config_file_path = _FLAGS['config_file_path'].value + except Exception: + config_file_path = DEFAULT_CONFIG_JSON + + # 1. If it's a local file path (or doesn't start with gs://), try loading it + if config_file_path and not config_file_path.startswith('gs://'): + config = _load_local_config(config_file_path) + if config and 'URLS_CONFIG' in config: + logging.info(f"Loaded config from local path: {config_file_path}") + return config + logging.warning( + f"Could not load config from local file '{config_file_path}'") + + # 2. If it's a GCS path, try loading from GCS + if config_file_path and config_file_path.startswith('gs://'): + try: + storage_client = storage.Client() + parts = config_file_path[5:].split('/', 1) + bucket_name = parts[0] + blob_name = parts[1] if len(parts) > 1 else '' + bucket = storage_client.bucket(bucket_name) + blob = bucket.blob(blob_name) + file_contents = blob.download_as_text() + if config_file_path.endswith('.json'): + return json.loads(file_contents) + # Deprecated: Legacy Python config (.py) via exec(). + # All imports should standardize on version-controlled JSON configuration. + logging.warning( + f"DEPRECATION WARNING: Loading Python configuration via exec() " + f"from '{config_file_path}' is deprecated and will be removed. " + "Please migrate to JSON format ('configs.json').") + local_vars = {} + exec(file_contents, {}, local_vars) + return local_vars + except Exception as e: + logging.warning( + f"Cannot extract url and related configs from GCS '{config_file_path}': {e}. " + "Falling back to local configuration.") + + # 3. Fall back to local configs.json in SCRIPT_DIR + config = _load_local_config(DEFAULT_CONFIG_JSON) + if config and 'URLS_CONFIG' in config: + logging.info( + f"Loaded fallback configuration from {DEFAULT_CONFIG_JSON}") + return config + + logging.error( + "Cannot extract url and related configs: all sources failed.") + raise RuntimeError( + "Cannot extract url and related configs: all sources failed.") + + +def download_files(URL_CONFIG, session=None, delay=0.5): + if not URL_CONFIG: + logging.warning("No URL configurations provided to download.") + return [] + + if session is None: + session = create_retry_session() + try: + logging.info( + f"Warming up session with RBI portal cookies from {RBI_PORTAL_URL}" + ) + session.get(RBI_PORTAL_URL, timeout=30) + except Exception as e: + logging.warning( + f"Could not warm up session from {RBI_PORTAL_URL}: {e}") + + failed_downloads = [] + + for config in URL_CONFIG: + config_url = config.get("url") + category_name = config.get("category") + file_name = config.get("filename") + if not config_url or not category_name or not file_name: + logging.warning(f"Skipping incomplete config: {config}") + failed_downloads.append(file_name or str(config)) + continue + + target_dir = os.path.join(INPUT_DIR, category_name) + os.makedirs(target_dir, exist_ok=True) + file_path = os.path.join(target_dir, file_name) + + force_download = False + try: + force_download = flags.FLAGS.force_download + except (flags.UnparsedFlagAccessError, AttributeError): + pass + + if os.path.exists(file_path): + if not force_download and os.path.getsize(file_path) > 0: + with open(file_path, 'rb') as f: + magic = f.read(4) + if magic == XLSX_ZIP_SIGNATURE: + logging.info(f"Skipping existing valid file: {file_name}") + continue + try: + os.remove(file_path) + except OSError: + pass + elif not force_download and os.path.getsize(file_path) == 0: + try: + os.remove(file_path) + except OSError: + pass + + try: + logging.info(f"Attempting GET request to: {config_url}") + response = session.get(config_url, timeout=45) + + if response.status_code == 404: + logging.error( + f"Table not found (HTTP 404) at {config_url} for {file_name}" + ) + failed_downloads.append(file_name) + continue + + response.raise_for_status() + content = response.content + + if not content: + logging.error( + f"Empty response received for {file_name} from {config_url}" + ) + failed_downloads.append(file_name) + continue + + if not content.startswith(XLSX_ZIP_SIGNATURE): + snippet = content[:50].decode('utf-8', errors='replace') + logging.error( + f"Invalid file format received for {file_name} from {config_url}: " + f"expected XLSX ZIP signature, got: {snippet!r}") + failed_downloads.append(file_name) + continue + + # Write atomically to a temporary directory, then move into place + with tempfile.TemporaryDirectory(dir=target_dir) as tmp_dir: + temp_path = os.path.join(tmp_dir, file_name) + with open(temp_path, 'wb') as tmp_file: + tmp_file.write(content) + os.replace(temp_path, file_path) + logging.info( + f"Downloaded the file {file_name} successfully ({len(content)} bytes)." + ) + + if delay > 0: + time.sleep(delay) + + except Exception as e: + logging.error( + f"Failed to download table {file_name} from {config_url}: {e}") + failed_downloads.append(file_name) + + return failed_downloads + + +def _apply_map(df, func): + """Version-agnostic element-wise DataFrame mapping + (uses df.map if available, else df.applymap). + """ + if hasattr(df, 'map'): + return df.map(func) + return df.applymap(func) def preprocess_files(directory_path): if not os.path.isdir(directory_path): - logging.info(f"Error: Directory not found at '{directory_path}'") - return - + logging.error(f"Error: Directory not found at '{directory_path}'") + return [directory_path] + xlsx_files = [f for f in os.listdir(directory_path) if f.endswith('.xlsx')] if not xlsx_files: - logging.info(f"No XLSX files found in the directory: {directory_path}") - return + logging.error( + f"No XLSX files found in the directory: {directory_path}") + return [directory_path] + + logging.info( + f"Found {len(xlsx_files)} XLSX files to process in '{directory_path}'." + ) - logging.info(f"Found {len(xlsx_files)} XLSX files to process in '{directory_path}'.") + failed_files = [] - for file_name in xlsx_files: + for file_name in sorted(xlsx_files): file_path = os.path.join(directory_path, file_name) logging.info(f"Processing file: {file_path}") try: - all_sheets_data = pd.read_excel(file_path, sheet_name=None, header=None) - with pd.ExcelWriter(file_path, engine='openpyxl') as writer: - for sheet_name, df in all_sheets_data.items(): - df = df.map(lambda x: str(x).replace('*', '').replace('@', '').strip()) - mask = df.eq('State/Union Territory').any(axis=1) - state_positions = df[mask] == 'State/Union Territory' - - def safe_to_numeric(val): - val_str = str(val) - if val_str.isdigit(): - return int(val_str) - try: - float_val = float(val_str) - return float_val - except ValueError: - return val - - df_num = df[mask].applymap(safe_to_numeric) - converted = df_num.mask(state_positions, 'State/Union Territory') + all_sheets_data = pd.read_excel(file_path, + sheet_name=None, + header=None, + engine='openpyxl') + + def clean_cell(val): + if pd.isna(val): + return val + if isinstance(val, (int, float)): + return val + val_str = str(val).replace('*', '').replace('@', '').strip() + if not val_str or val_str.lower() == 'nan': + return float('nan') + return val_str + + def safe_to_numeric(val): + if pd.isna(val): + return val + if isinstance(val, int) and not isinstance(val, bool): + return val + if isinstance(val, float): + return int(val) if val.is_integer() else val + val_str = str(val).strip() + if not val_str or val_str.lower() == 'nan': + return float('nan') + if val_str.isdigit(): + return int(val_str) + try: + float_val = float(val_str) + if float_val.is_integer(): + return int(float_val) + return float_val + except ValueError: + return val + + def is_state_header(val): + if pd.isna(val): + return False + return bool( + re.search( + r'state\s*/\s*(?:union(?:\s*territory)?|u\.?t\.?\b)', + str(val), re.IGNORECASE)) + + for sheet_name, df in all_sheets_data.items(): + df = _apply_map(df, clean_cell) + is_state_cell = _apply_map(df, is_state_header) + mask = is_state_cell.any(axis=1) + + if mask.any(): + state_positions = is_state_cell.loc[mask] + df_num = _apply_map(df[mask], safe_to_numeric) + converted = df_num.mask(state_positions, df[mask]) + df = df.astype(object) df.loc[mask, :] = converted - df.to_excel(writer, sheet_name=sheet_name, index=False, header=False) + all_sheets_data[sheet_name] = df + + with tempfile.TemporaryDirectory(dir=directory_path) as output_dir: + output_path = os.path.join(output_dir, file_name) + with pd.ExcelWriter(output_path, engine='openpyxl') as writer: + for sheet_name, df in all_sheets_data.items(): + df.to_excel(writer, + sheet_name=sheet_name, + index=False, + header=False) + os.replace(output_path, file_path) + logging.info( + f"Successfully processed {file_name} ({len(all_sheets_data)} sheet(s))." + ) except Exception as e: - logging.fatal(f"Error processing {file_name}: {e}") + logging.error(f"Error processing {file_name}: {e}") + failed_files.append(file_name) + + if failed_files: + logging.error( + f"Failed to process {len(failed_files)} file(s) in '{directory_path}': {failed_files}" + ) + return failed_files logging.info("All specified XLSX files have been processed.") + return [] -def main(_): +def main(argv): + if len(argv) > 1: + raise app.UsageError('Too many command-line arguments.') configs = reads_config_file() RBI_URL = configs['URLS_CONFIG'] - download_files(RBI_URL) + failed_downloads = download_files(RBI_URL) + if failed_downloads: + logging.fatal( + f"Download failed for {len(failed_downloads)} table(s): {failed_downloads}" + ) logging.info("Download process Completed successfully") - directories = ['agriculture', 'environment', 'infrastructure', 'price_and_wages'] + directories = [ + 'agriculture', 'environment', 'infrastructure', 'price_and_wages' + ] + failed_preprocesses = [] for directory in directories: - preprocess_files(os.path.join(INPUT_DIR, directory)) + failed = preprocess_files(os.path.join(INPUT_DIR, directory)) + if failed: + failed_preprocesses.extend(failed) + if failed_preprocesses: + logging.fatal( + f"Preprocessing failed for {len(failed_preprocesses)} file(s): {failed_preprocesses}" + ) logging.info("Pre-process Completed successfully") -if __name__ == "__main__": +if __name__ == "__main__": app.run(main) diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_download_test.py b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_download_test.py new file mode 100644 index 0000000000..161ad31b36 --- /dev/null +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_download_test.py @@ -0,0 +1,394 @@ +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import json +import os +import pathlib +import sys +import tempfile +from unittest import mock + +from absl import app +from absl.testing import absltest, flagsaver +import pandas as pd +import requests + +_SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__)) +sys.path.append(_SCRIPT_DIR) + +import rbi_download + + +class PreprocessFilesTest(absltest.TestCase): + + def test_preserves_workbook_and_original_error(self): + with tempfile.TemporaryDirectory() as directory: + workbook = pathlib.Path(directory) / 'source.xlsx' + workbook.write_bytes(b'original workbook') + sheets = {'Sheet1': pd.DataFrame([['State/Union Territory']])} + + with mock.patch.object( + rbi_download.pd, 'read_excel', return_value=sheets), \ + mock.patch.object( + rbi_download, + '_apply_map', + side_effect=ValueError('transform failed')), \ + mock.patch.object(rbi_download.logging, 'error') as mock_error, \ + mock.patch.object(rbi_download.logging, 'fatal') as mock_fatal: + failed = rbi_download.preprocess_files(directory) + + self.assertEqual(workbook.read_bytes(), b'original workbook') + mock_error.assert_any_call( + 'Error processing source.xlsx: transform failed') + self.assertEqual(mock_error.call_count, 2) + mock_fatal.assert_not_called() + self.assertEqual(failed, ['source.xlsx']) + + def test_preprocess_files_converts_numeric_headers(self): + with tempfile.TemporaryDirectory() as directory: + file_path = pathlib.Path(directory) / 'source.xlsx' + initial_df = pd.DataFrame( + [['State/Union Territory', '2015*', '2016@', '2017-18', 2018.0], + ['Andhra Pradesh', '10.5', '20.0', '30.5', '40.0']]) + with pd.ExcelWriter(file_path, engine='openpyxl') as writer: + initial_df.to_excel(writer, + sheet_name='Sheet1', + index=False, + header=False) + + rbi_download.preprocess_files(directory) + + processed = pd.read_excel(file_path, + sheet_name='Sheet1', + header=None) + self.assertEqual(processed.iloc[0, 0], 'State/Union Territory') + self.assertEqual(processed.iloc[0, 1], 2015) + self.assertEqual(processed.iloc[0, 2], 2016) + self.assertEqual(processed.iloc[0, 3], '2017-18') + self.assertEqual(processed.iloc[0, 4], 2018) + self.assertEqual(processed.iloc[1, 0], 'Andhra Pradesh') + + def test_preprocess_files_preserves_nan(self): + with tempfile.TemporaryDirectory() as directory: + file_path = pathlib.Path(directory) / 'source.xlsx' + initial_df = pd.DataFrame( + [['State/Union Territory', '2015*', None], + ['Andhra Pradesh', None, '30.5']]) + with pd.ExcelWriter(file_path, engine='openpyxl') as writer: + initial_df.to_excel(writer, + sheet_name='Sheet1', + index=False, + header=False) + + rbi_download.preprocess_files(directory) + + processed = pd.read_excel(file_path, + sheet_name='Sheet1', + header=None) + self.assertTrue(pd.isna(processed.iloc[0, 2])) + self.assertTrue(pd.isna(processed.iloc[1, 1])) + + def test_preprocess_files_error_on_missing_dir(self): + with mock.patch.object(rbi_download.logging, 'error') as mock_error: + failed = rbi_download.preprocess_files( + '/non/existent/directory/path') + self.assertEqual(failed, ['/non/existent/directory/path']) + mock_error.assert_called_once() + self.assertIn("Directory not found", mock_error.call_args[0][0]) + + def test_preprocess_files_error_on_empty_dir(self): + with tempfile.TemporaryDirectory() as directory: + with mock.patch.object(rbi_download.logging, 'error') as mock_error: + failed = rbi_download.preprocess_files(directory) + self.assertEqual(failed, [directory]) + mock_error.assert_called_once() + self.assertIn("No XLSX files found", mock_error.call_args[0][0]) + + def test_preprocess_files_handles_flexible_state_header(self): + with tempfile.TemporaryDirectory() as directory: + file_path = pathlib.Path(directory) / 'source.xlsx' + initial_df = pd.DataFrame([['State / Union Territory', '2018-19'], + ['State/Union Territory', '2019-20'], + ['Karnataka', '120.5']]) + with pd.ExcelWriter(file_path, engine='openpyxl') as writer: + initial_df.to_excel(writer, + sheet_name='Sheet1', + index=False, + header=False) + + rbi_download.preprocess_files(directory) + + processed = pd.read_excel(file_path, + sheet_name='Sheet1', + header=None) + self.assertEqual(processed.iloc[0, 0], 'State / Union Territory') + self.assertEqual(processed.iloc[1, 0], 'State/Union Territory') + self.assertEqual(processed.iloc[0, 1], '2018-19') + self.assertEqual(processed.iloc[1, 1], '2019-20') + self.assertEqual(processed.iloc[2, 0], 'Karnataka') + self.assertEqual(processed.iloc[2, 1], '120.5') + + def test_preprocess_files_avoids_literal_nan_and_preserves_numeric(self): + with tempfile.TemporaryDirectory() as directory: + file_path = pathlib.Path(directory) / 'source.xlsx' + initial_df = pd.DataFrame( + [['State/Union Territory', '2020', '2021', '2022'], + ['Bihar', 100, 200.5, ' '], ['Assam', 'nan', 'NAN', '300']]) + with pd.ExcelWriter(file_path, engine='openpyxl') as writer: + initial_df.to_excel(writer, + sheet_name='Sheet1', + index=False, + header=False) + + rbi_download.preprocess_files(directory) + + processed = pd.read_excel(file_path, + sheet_name='Sheet1', + header=None) + # Empty/whitespace and 'nan' strings become NaN, NOT literal string 'nan' + self.assertTrue(pd.isna(processed.iloc[1, 3])) + self.assertTrue(pd.isna(processed.iloc[2, 1])) + self.assertTrue(pd.isna(processed.iloc[2, 2])) + # Numeric values preserved + self.assertEqual(processed.iloc[1, 1], 100) + self.assertEqual(processed.iloc[1, 2], 200.5) + + +class ReadsConfigFileTest(absltest.TestCase): + + def test_reads_local_config_file(self): + with tempfile.NamedTemporaryFile('w', suffix='.json', + delete=False) as f: + json.dump( + { + 'URLS_CONFIG': [{ + 'url': 'http://example.com/test.xlsx', + 'category': 'test', + 'filename': 'test.xlsx' + }] + }, f) + temp_path = f.name + try: + with flagsaver.flagsaver(config_file_path=temp_path): + configs = rbi_download.reads_config_file() + self.assertIn('URLS_CONFIG', configs) + self.assertEqual(len(configs['URLS_CONFIG']), 1) + self.assertEqual(configs['URLS_CONFIG'][0]['filename'], + 'test.xlsx') + finally: + os.remove(temp_path) + + def test_falls_back_to_local_configs_json(self): + with flagsaver.flagsaver( + config_file_path='gs://nonexistent_bucket/configs.json'), \ + mock.patch.object( + rbi_download.storage, + 'Client', + side_effect=Exception('GCS unavailable')): + configs = rbi_download.reads_config_file() + self.assertIn('URLS_CONFIG', configs) + self.assertGreater(len(configs['URLS_CONFIG']), 0) + + def test_reads_gcs_python_config_logs_deprecation(self): + mock_blob = mock.MagicMock() + mock_blob.download_as_text.return_value = ( + 'URLS_CONFIG = [{"url": "http://example.com/test.xlsx", ' + '"category": "cat", "filename": "test.xlsx"}]\n' + ) + mock_bucket = mock.MagicMock() + mock_bucket.blob.return_value = mock_blob + mock_client = mock.MagicMock() + mock_client.bucket.return_value = mock_bucket + + with flagsaver.flagsaver( + config_file_path='gs://bucket/legacy_configs.py'), \ + mock.patch.object( + rbi_download.storage, 'Client', return_value=mock_client), \ + mock.patch.object( + rbi_download.logging, 'warning') as mock_warning: + configs = rbi_download.reads_config_file() + self.assertIn('URLS_CONFIG', configs) + deprecation_logged = any( + 'DEPRECATION WARNING' in str(call[0][0]) + for call in mock_warning.call_args_list + ) + self.assertTrue(deprecation_logged) + + +class DownloadFilesTest(absltest.TestCase): + + def test_skips_existing_valid_file(self): + with tempfile.TemporaryDirectory() as temp_dir: + cat_dir = os.path.join(temp_dir, 'agri') + os.makedirs(cat_dir, exist_ok=True) + existing_file = os.path.join(cat_dir, 'existing.xlsx') + with open(existing_file, 'wb') as f: + f.write(b'PK\x03\x04valid_zip_content') + + configs = [{ + 'url': 'http://example.com/existing.xlsx', + 'category': 'agri', + 'filename': 'existing.xlsx' + }] + mock_session = mock.MagicMock() + with mock.patch.object(rbi_download, 'INPUT_DIR', temp_dir): + rbi_download.download_files(configs, + session=mock_session, + delay=0) + mock_session.get.assert_not_called() + + def test_redownloads_invalid_existing_file(self): + with tempfile.TemporaryDirectory() as temp_dir: + cat_dir = os.path.join(temp_dir, 'agri') + os.makedirs(cat_dir, exist_ok=True) + corrupt_file = os.path.join(cat_dir, 'corrupt.xlsx') + with open(corrupt_file, 'wb') as f: + f.write(b'error page') + + configs = [{ + 'url': 'http://example.com/corrupt.xlsx', + 'category': 'agri', + 'filename': 'corrupt.xlsx' + }] + mock_session = mock.MagicMock() + mock_response = mock.MagicMock() + mock_response.status_code = 200 + mock_response.content = b'PK\x03\x04valid_content' + mock_session.get.return_value = mock_response + + with mock.patch.object(rbi_download, 'INPUT_DIR', temp_dir): + rbi_download.download_files(configs, + session=mock_session, + delay=0) + mock_session.get.assert_called_once() + self.assertEqual( + pathlib.Path(corrupt_file).read_bytes(), + b'PK\x03\x04valid_content') + + def test_download_files_uses_browser_headers_in_session(self): + session = rbi_download.create_retry_session() + headers = session.headers + self.assertIn('User-Agent', headers) + self.assertIn('Mozilla', headers['User-Agent']) + self.assertIn('Referer', headers) + self.assertIn('rbi.org.in', headers['Referer']) + + def test_rejects_html_error_response(self): + with tempfile.TemporaryDirectory() as temp_dir: + configs = [{ + 'url': 'http://example.com/blocked.xlsx', + 'category': 'agri', + 'filename': 'blocked.xlsx' + }] + mock_session = mock.MagicMock() + mock_response = mock.MagicMock() + mock_response.status_code = 200 + mock_response.content = b'Access Denied' + mock_session.get.return_value = mock_response + + with mock.patch.object(rbi_download, 'INPUT_DIR', temp_dir): + rbi_download.download_files(configs, + session=mock_session, + delay=0) + target_file = pathlib.Path(temp_dir) / 'agri' / 'blocked.xlsx' + self.assertFalse(target_file.exists()) + + def test_treats_404_as_failure(self): + with tempfile.TemporaryDirectory() as temp_dir: + configs = [{ + 'url': 'http://example.com/missing.xlsx', + 'category': 'agri', + 'filename': 'missing.xlsx' + }] + mock_session = mock.MagicMock() + mock_response = mock.MagicMock() + mock_response.status_code = 404 + mock_session.get.return_value = mock_response + + with mock.patch.object(rbi_download, 'INPUT_DIR', temp_dir): + failed = rbi_download.download_files(configs, + session=mock_session, + delay=0) + self.assertEqual(failed, ['missing.xlsx']) + target_file = pathlib.Path(temp_dir) / 'agri' / 'missing.xlsx' + self.assertFalse(target_file.exists()) + + def test_isolates_download_failures(self): + with tempfile.TemporaryDirectory() as temp_dir: + configs = [ + { + 'url': 'http://example.com/fail.xlsx', + 'category': 'agri', + 'filename': 'fail.xlsx' + }, + { + 'url': 'http://example.com/success.xlsx', + 'category': 'agri', + 'filename': 'success.xlsx' + }, + ] + mock_session = mock.MagicMock() + mock_success = mock.MagicMock() + mock_success.status_code = 200 + mock_success.content = b'PK\x03\x04success content' + mock_session.get.side_effect = [ + requests.exceptions.RequestException('404 Not Found'), + mock_success + ] + + with mock.patch.object(rbi_download, 'INPUT_DIR', temp_dir): + failed = rbi_download.download_files(configs, + session=mock_session, + delay=0) + + success_file = pathlib.Path(temp_dir) / 'agri' / 'success.xlsx' + self.assertTrue(success_file.exists()) + self.assertEqual(success_file.read_bytes(), + b'PK\x03\x04success content') + self.assertEqual(failed, ['fail.xlsx']) + + +class MainExecutionTest(absltest.TestCase): + + def test_main_fatal_when_download_fails(self): + with mock.patch.object( + rbi_download, 'reads_config_file', return_value={'URLS_CONFIG': []}), \ + mock.patch.object( + rbi_download, 'download_files', return_value=['failed_table.xlsx']), \ + mock.patch.object(rbi_download.logging, 'fatal', side_effect=SystemExit) as mock_fatal: + with self.assertRaises(SystemExit): + rbi_download.main(['rbi_download']) + mock_fatal.assert_called_once() + self.assertIn('failed_table.xlsx', mock_fatal.call_args[0][0]) + + def test_main_fatal_when_preprocessing_fails(self): + with mock.patch.object( + rbi_download, 'reads_config_file', return_value={'URLS_CONFIG': []}), \ + mock.patch.object( + rbi_download, 'download_files', return_value=[]), \ + mock.patch.object( + rbi_download, 'preprocess_files', side_effect=[['broken.xlsx'], [], [], []]), \ + mock.patch.object(rbi_download.logging, 'fatal', side_effect=SystemExit) as mock_fatal: + with self.assertRaises(SystemExit): + rbi_download.main(['rbi_download']) + mock_fatal.assert_called_once() + self.assertIn('broken.xlsx', mock_fatal.call_args[0][0]) + + def test_main_usage_error_too_many_args(self): + with self.assertRaises(app.UsageError): + rbi_download.main(['rbi_download', 'unexpected_argument']) + + +if __name__ == '__main__': + absltest.main() diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv index a67a4fa4f1..40b2a5fc5e 100644 --- a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/rbi_places_resolver.csv @@ -8,7 +8,7 @@ Chandigarh,wikidataId/Q43433 Chhattisgarh,wikidataId/Q1168 Dadra & Nagar Haveli,wikidataId/Q46107 Daman & Diu,wikidataId/Q66710 -NCT of Delhi,wikidataId/Q1352 +NCT of Delhi,wikidataId/Q1353 Delhi,wikidataId/Q1353 Goa,wikidataId/Q1171 Gujarat,wikidataId/Q1061 @@ -38,9 +38,10 @@ Uttarakhand,wikidataId/Q1499 West Bengal,wikidataId/Q1356 ALL INDIA,country/IND All-India,country/IND +All India,country/IND Ladakh,wikidataId/Q200667 -Kolkata**,wikidataId/Q1348 -Chennai**,wikidataId/Q1352 +Kolkata,wikidataId/Q1348 +Chennai,wikidataId/Q1352 Mumbai,wikidataId/Q1156 TamilNadu,wikidataId/Q1445 -Andaman & NicobarIslands,wikidataId/Q40888 \ No newline at end of file +Andaman & NicobarIslands,wikidataId/Q40888 diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/run.sh b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/run.sh index 29f9bdd623..927ce4b8da 100644 --- a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/run.sh +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/run.sh @@ -1,44 +1,266 @@ #!/bin/bash -set -e +set -euo pipefail SCRIPT_PATH=$(realpath "$(dirname "$0")") +COUNTERS_PATH="$SCRIPT_PATH/counters" +mkdir -p "$COUNTERS_PATH" -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/agriculture/*.xlsx --pv_map=$SCRIPT_PATH/agriculture_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/agriculture/agriculture_output || { echo "Error: Processing Agriculture files failed!"; exit 1; } +# Existing statvar MCF path with local fallback +LOCAL_MCF="$SCRIPT_PATH/stat_vars.mcf" +if [ -f "$LOCAL_MCF" ]; then + EXISTING_MCF="$LOCAL_MCF" +else + EXISTING_MCF="${EXISTING_STATVAR_MCF:-gs://unresolved_mcf/scripts/statvar/stat_vars.mcf}" +fi -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/environment/state_wise_forest_cover.xlsx --pv_map=$SCRIPT_PATH/environment_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/environment/state_wise_forest_cover_output || { echo "Error: Processing State-wise Forest Cover failed!"; exit 1; } +PROCESSOR="$SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py" +IN_DIR="$SCRIPT_PATH/input_files" +OUT_DIR="$SCRIPT_PATH/output_files" +IN_ENV="$IN_DIR/environment" +IN_INFRA="$IN_DIR/infrastructure" +OUT_ENV="$OUT_DIR/environment" +OUT_INFRA="$OUT_DIR/infrastructure" +PLACES_CSV="$SCRIPT_PATH/rbi_places_resolver.csv" +META_RBI="$SCRIPT_PATH/rbi_metadata.csv" +META_INFRA="$SCRIPT_PATH/infrastructure_metadata.csv" +META_SDG="$SCRIPT_PATH/environment_sdg_metadata.csv" -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/environment/state_wise_tree_cover.xlsx --pv_map=$SCRIPT_PATH/environment_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/environment/state_wise_tree_cover_output || { echo "Error: Processing State-wise Tree Cover failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_DIR/agriculture/*.xlsx" \ + --pv_map="$SCRIPT_PATH/agriculture_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/agriculture_counters.csv" \ + --output_path="$OUT_DIR/agriculture/agriculture_output" \ + || { + echo "Error: Processing Agriculture files failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/environment/sub_division_wise_annual_rainfall.xlsx --pv_map=$SCRIPT_PATH/environment_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/environment/sub_division_wise_annual_rainfall_output || { echo "Error: Processing Sub-Division-wise Annual Rainfall failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_ENV/state_wise_forest_cover.xlsx" \ + --pv_map="$SCRIPT_PATH/environment_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_forest_cover_counters.csv" \ + --output_path="$OUT_ENV/state_wise_forest_cover_output" \ + || { + echo "Error: Processing State-wise Forest Cover failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/environment/state_wise_expenditure_on_relief_on_natural_calamities.xlsx --pv_map=$SCRIPT_PATH/environment_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/environment/state_wise_expenditure_on_relief_on_natural_calamities_output || { echo "Error: Processing State-wise Expenditure on Relief on Natural Calamities failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_ENV/state_wise_tree_cover.xlsx" \ + --pv_map="$SCRIPT_PATH/environment_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_tree_cover_counters.csv" \ + --output_path="$OUT_ENV/state_wise_tree_cover_output" \ + || { + echo "Error: Processing State-wise Tree Cover failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/environment/state_wise_sustainable_development_goals_score_SDGs.xlsx --pv_map=$SCRIPT_PATH/environment_pvmap.csv --config_file=$SCRIPT_PATH/environment_sdg_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/environment/state_wise_sdg_score_output || { echo "Error: Processing State-wise Sustainable Development Goals Score failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_ENV/sub_division_wise_annual_rainfall.xlsx" \ + --pv_map="$SCRIPT_PATH/environment_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/sub_division_wise_annual_rainfall_counters.csv" \ + --output_path="$OUT_ENV/sub_division_wise_annual_rainfall_output" \ + || { + echo "Error: Processing Sub-Division-wise Annual Rainfall failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/price_and_wages/*.xlsx --pv_map=$SCRIPT_PATH/price_wages_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/price_and_wages/price_and_wages_output || { echo "Error: Processing price_and_wages files failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_ENV/state_wise_expenditure_on_relief_on_natural_calamities.xlsx" \ + --pv_map="$SCRIPT_PATH/environment_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters=\ + "$COUNTERS_PATH/state_wise_expenditure_on_relief_on_natural_calamities_counters.csv" \ + --output_path="$OUT_ENV/state_wise_expenditure_on_relief_on_natural_calamities_output" \ + || { + echo "Error: Processing State-wise Expenditure on Relief on Natural Calamities failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_per_capita_availability_of_power.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_per_capita_availability_of_power_output || { echo "Error: Processing State-wise Per Capita Availability of Power failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_ENV/state_wise_sustainable_development_goals_score_SDGs.xlsx" \ + --pv_map="$SCRIPT_PATH/environment_pvmap.csv" \ + --config_file="$META_SDG" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_sdg_score_counters.csv" \ + --output_path="$OUT_ENV/state_wise_sdg_score_output" \ + || { + echo "Error: Processing State-wise Sustainable Development Goals Score failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_availability_of_power.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_availability_of_power_output || { echo "Error: Processing State-wise Availability of Power files failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_DIR/price_and_wages/*.xlsx" \ + --pv_map="$SCRIPT_PATH/price_wages_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/price_and_wages_counters.csv" \ + --output_path="$OUT_DIR/price_and_wages/price_and_wages_output" \ + || { + echo "Error: Processing price_and_wages files failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_installed_capacity_of_power.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_installed_capacity_of_power_output || { echo "Error: Processing State-wise Installed Capacity of Power failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_per_capita_availability_of_power.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_per_capita_availability_of_power_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_per_capita_availability_of_power_output" \ + || { + echo "Error: Processing State-wise Per Capita Availability of Power failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_power_requirement.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_power_requirement_output || { echo "Error: Processing State-wise Power Requirement failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_availability_of_power.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_availability_of_power_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_availability_of_power_output" \ + || { + echo "Error: Processing State-wise Availability of Power files failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_length_of_national_highways.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/infrastructure_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_length_of_national_highways_output || { echo "Error: Processing State-wise Length of National Highways failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_installed_capacity_of_power.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_installed_capacity_of_power_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_installed_capacity_of_power_output" \ + || { + echo "Error: Processing State-wise Installed Capacity of Power failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_railway_route.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/infrastructure_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_railway_route_output || { echo "Error: Processing State-wise Railway Route failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_power_requirement.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_power_requirement_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_power_requirement_output" \ + || { + echo "Error: Processing State-wise Power Requirement failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_length_of_roads.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/infrastructure_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_length_of_roads_output || { echo "Error: Processing State-wise Tree Cover failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_length_of_national_highways.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_INFRA" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_length_of_national_highways_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_length_of_national_highways_output" \ + || { + echo "Error: Processing State-wise Length of National Highways failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_length_of_state_highways.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/infrastructure_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_length_of_state_highways_output || { echo "Error: Processing State-wise Length of State Highways failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_railway_route.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_INFRA" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_railway_route_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_railway_route_output" \ + || { + echo "Error: Processing State-wise Railway Route failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_electricity_transmission_distribution_losses.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_electricity_transmission_distribution_losses_output || { echo "Error: Processing State-wise Electricity Transmission & Distribution Losses failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_length_of_roads.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_INFRA" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_length_of_roads_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_length_of_roads_output" \ + || { + echo "Error: Processing State-wise Length of Roads failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_telephones_per_100_population.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_telephones_per_100_population_output || { echo "Error: Processing State-wise Telephones per 100 Population failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_length_of_state_highways.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_INFRA" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_length_of_state_highways_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_length_of_state_highways_output" \ + || { + echo "Error: Processing State-wise Length of State Highways failed!" + exit 1 + } -python3 $SCRIPT_PATH/../../../tools/statvar_importer/stat_var_processor.py --input_data=$SCRIPT_PATH/input_files/infrastructure/state_wise_road_constructed_under_PMGSY.xlsx --pv_map=$SCRIPT_PATH/infrastructure_pvmap.csv --config_file=$SCRIPT_PATH/rbi_metadata.csv --places_resolved_csv=$SCRIPT_PATH/rbi_places_resolver.csv --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf --output_path=$SCRIPT_PATH/output_files/infrastructure/state_wise_road_constructed_under_pmgsy_output || { echo "Error: Processing State-wise Road Constructed under PMGSY failed!"; exit 1; } +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_electricity_transmission_distribution_losses.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters=\ + "$COUNTERS_PATH/state_wise_electricity_transmission_distribution_losses_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_electricity_transmission_distribution_losses_output" \ + || { + echo "Error: Processing State-wise Electricity Transmission & Distribution Losses failed!" + exit 1 + } -echo "All processing steps completed successfully." -exit 0 +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_telephones_per_100_population.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_telephones_per_100_population_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_telephones_per_100_population_output" \ + || { + echo "Error: Processing State-wise Telephones per 100 Population failed!" + exit 1 + } + +python3 "$PROCESSOR" \ + --input_data="$IN_INFRA/state_wise_road_constructed_under_PMGSY.xlsx" \ + --pv_map="$SCRIPT_PATH/infrastructure_pvmap.csv" \ + --config_file="$META_RBI" \ + --places_resolved_csv="$PLACES_CSV" \ + --existing_statvar_mcf="$EXISTING_MCF" \ + --output_counters="$COUNTERS_PATH/state_wise_road_constructed_under_pmgsy_counters.csv" \ + --output_path="$OUT_INFRA/state_wise_road_constructed_under_pmgsy_output" \ + || { + echo "Error: Processing State-wise Road Constructed under PMGSY failed!" + exit 1 + } +EXIT_STATUS=$? +echo "All processing steps completed successfully." +exit $EXIT_STATUS diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/testdata/environment_input.xlsx b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/testdata/environment_input.xlsx index 23cb3f9b2e..6acf6eed9c 100644 Binary files a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/testdata/environment_input.xlsx and b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/testdata/environment_input.xlsx differ diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/testdata/environment_output.csv b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/testdata/environment_output.csv index a03144fc2e..1b5c38091f 100644 --- a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/testdata/environment_output.csv +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/testdata/environment_output.csv @@ -1 +1,673 @@ observationAbout,observationDate,variableMeasured,value,unit,observationPeriod +dcid:wikidataId/Q40888,2000,dcid:Annual_Rainfall_Place,2763.2,Millimeter, +dcid:wikidataId/Q40888,2001,dcid:Annual_Rainfall_Place,3080.9,Millimeter, +dcid:wikidataId/Q40888,2002,dcid:Annual_Rainfall_Place,2620.2,Millimeter, +dcid:wikidataId/Q40888,2003,dcid:Annual_Rainfall_Place,2355.9,Millimeter, +dcid:wikidataId/Q40888,2004,dcid:Annual_Rainfall_Place,2460.1,Millimeter, +dcid:wikidataId/Q40888,2005,dcid:Annual_Rainfall_Place,2954.7,Millimeter, +dcid:wikidataId/Q40888,2006,dcid:Annual_Rainfall_Place,2404.7,Millimeter, +dcid:wikidataId/Q40888,2007,dcid:Annual_Rainfall_Place,2748,Millimeter, +dcid:HaryanaDelhiChandigarh,2000,dcid:Annual_Rainfall_Place,384.4,Millimeter, +dcid:HaryanaDelhiChandigarh,2001,dcid:Annual_Rainfall_Place,485.5,Millimeter, +dcid:HaryanaDelhiChandigarh,2002,dcid:Annual_Rainfall_Place,357.2,Millimeter, +dcid:HaryanaDelhiChandigarh,2003,dcid:Annual_Rainfall_Place,597.6,Millimeter, +dcid:HaryanaDelhiChandigarh,2004,dcid:Annual_Rainfall_Place,487.3,Millimeter, +dcid:HaryanaDelhiChandigarh,2005,dcid:Annual_Rainfall_Place,519.5,Millimeter, +dcid:HaryanaDelhiChandigarh,2006,dcid:Annual_Rainfall_Place,330.9,Millimeter, +dcid:HaryanaDelhiChandigarh,2007,dcid:Annual_Rainfall_Place,446.6,Millimeter, +dcid:wikidataId/Q40888,1987,dcid:Area_LandCover_Forest,7601,SquareKilometer, +dcid:wikidataId/Q40888,1989,dcid:Area_LandCover_Forest,7622,SquareKilometer, +dcid:wikidataId/Q40888,1991,dcid:Area_LandCover_Forest,7622,SquareKilometer, +dcid:wikidataId/Q40888,1993,dcid:Area_LandCover_Forest,7624,SquareKilometer, +dcid:wikidataId/Q40888,1995,dcid:Area_LandCover_Forest,7615,SquareKilometer, +dcid:wikidataId/Q40888,1997,dcid:Area_LandCover_Forest,7613,SquareKilometer, +dcid:wikidataId/Q40888,1999,dcid:Area_LandCover_Forest,7606,SquareKilometer, +dcid:wikidataId/Q40888,2001,dcid:Area_LandCover_Forest,6930,SquareKilometer, +dcid:wikidataId/Q40888,2003,dcid:Area_LandCover_Forest,6964,SquareKilometer, +dcid:wikidataId/Q40888,2005,dcid:Area_LandCover_Forest,6663,SquareKilometer, +dcid:wikidataId/Q1159,1987,dcid:Area_LandCover_Forest,49573,SquareKilometer, +dcid:wikidataId/Q1159,1989,dcid:Area_LandCover_Forest,47290,SquareKilometer, +dcid:wikidataId/Q1159,1991,dcid:Area_LandCover_Forest,47290,SquareKilometer, +dcid:wikidataId/Q1159,1993,dcid:Area_LandCover_Forest,47256,SquareKilometer, +dcid:wikidataId/Q1159,1995,dcid:Area_LandCover_Forest,47112,SquareKilometer, +dcid:wikidataId/Q1159,1997,dcid:Area_LandCover_Forest,43290,SquareKilometer, +dcid:wikidataId/Q1159,1999,dcid:Area_LandCover_Forest,44229,SquareKilometer, +dcid:wikidataId/Q1159,2001,dcid:Area_LandCover_Forest,44637,SquareKilometer, +dcid:wikidataId/Q1159,2003,dcid:Area_LandCover_Forest,44419,SquareKilometer, +dcid:wikidataId/Q1159,2005,dcid:Area_LandCover_Forest,45231,SquareKilometer, +dcid:wikidataId/Q1162,1987,dcid:Area_LandCover_Forest,64132,SquareKilometer, +dcid:wikidataId/Q1162,1989,dcid:Area_LandCover_Forest,69002,SquareKilometer, +dcid:wikidataId/Q1162,1991,dcid:Area_LandCover_Forest,68757,SquareKilometer, +dcid:wikidataId/Q1162,1993,dcid:Area_LandCover_Forest,68661,SquareKilometer, +dcid:wikidataId/Q1162,1995,dcid:Area_LandCover_Forest,68621,SquareKilometer, +dcid:wikidataId/Q1162,1997,dcid:Area_LandCover_Forest,68602,SquareKilometer, +dcid:wikidataId/Q1162,1999,dcid:Area_LandCover_Forest,68847,SquareKilometer, +dcid:wikidataId/Q1162,2001,dcid:Area_LandCover_Forest,68045,SquareKilometer, +dcid:wikidataId/Q1162,2003,dcid:Area_LandCover_Forest,68019,SquareKilometer, +dcid:wikidataId/Q1162,2005,dcid:Area_LandCover_Forest,67472,SquareKilometer, +dcid:wikidataId/Q1164,1987,dcid:Area_LandCover_Forest,25160,SquareKilometer, +dcid:wikidataId/Q1164,1989,dcid:Area_LandCover_Forest,24832,SquareKilometer, +dcid:wikidataId/Q1164,1991,dcid:Area_LandCover_Forest,24751,SquareKilometer, +dcid:wikidataId/Q1164,1993,dcid:Area_LandCover_Forest,24508,SquareKilometer, +dcid:wikidataId/Q1164,1995,dcid:Area_LandCover_Forest,24061,SquareKilometer, +dcid:wikidataId/Q1164,1997,dcid:Area_LandCover_Forest,23824,SquareKilometer, +dcid:wikidataId/Q1164,1999,dcid:Area_LandCover_Forest,23688,SquareKilometer, +dcid:wikidataId/Q1164,2001,dcid:Area_LandCover_Forest,27714,SquareKilometer, +dcid:wikidataId/Q1164,2003,dcid:Area_LandCover_Forest,27826,SquareKilometer, +dcid:wikidataId/Q1164,2005,dcid:Area_LandCover_Forest,27758,SquareKilometer, +dcid:wikidataId/Q1165,1987,dcid:Area_LandCover_Forest,28482,SquareKilometer, +dcid:wikidataId/Q1165,1989,dcid:Area_LandCover_Forest,26668,SquareKilometer, +dcid:wikidataId/Q1165,1991,dcid:Area_LandCover_Forest,26668,SquareKilometer, +dcid:wikidataId/Q1165,1993,dcid:Area_LandCover_Forest,26587,SquareKilometer, +dcid:wikidataId/Q1165,1995,dcid:Area_LandCover_Forest,26561,SquareKilometer, +dcid:wikidataId/Q1165,1997,dcid:Area_LandCover_Forest,4832,SquareKilometer, +dcid:wikidataId/Q1165,1999,dcid:Area_LandCover_Forest,4830,SquareKilometer, +dcid:wikidataId/Q1165,2001,dcid:Area_LandCover_Forest,5720,SquareKilometer, +dcid:wikidataId/Q1165,2003,dcid:Area_LandCover_Forest,5558,SquareKilometer, +dcid:wikidataId/Q1165,2005,dcid:Area_LandCover_Forest,6807,SquareKilometer, +dcid:wikidataId/Q43433,1987,dcid:Area_LandCover_Forest,2,SquareKilometer, +dcid:wikidataId/Q43433,1989,dcid:Area_LandCover_Forest,5,SquareKilometer, +dcid:wikidataId/Q43433,1991,dcid:Area_LandCover_Forest,5,SquareKilometer, +dcid:wikidataId/Q43433,1993,dcid:Area_LandCover_Forest,5,SquareKilometer, +dcid:wikidataId/Q43433,1995,dcid:Area_LandCover_Forest,7,SquareKilometer, +dcid:wikidataId/Q43433,1997,dcid:Area_LandCover_Forest,7,SquareKilometer, +dcid:wikidataId/Q43433,1999,dcid:Area_LandCover_Forest,7,SquareKilometer, +dcid:wikidataId/Q43433,2001,dcid:Area_LandCover_Forest,9,SquareKilometer, +dcid:wikidataId/Q43433,2003,dcid:Area_LandCover_Forest,15,SquareKilometer, +dcid:wikidataId/Q43433,2005,dcid:Area_LandCover_Forest,17,SquareKilometer, +dcid:wikidataId/Q1168,1997,dcid:Area_LandCover_Forest,56435,SquareKilometer, +dcid:wikidataId/Q1168,1999,dcid:Area_LandCover_Forest,56693,SquareKilometer, +dcid:wikidataId/Q1168,2001,dcid:Area_LandCover_Forest,56448,SquareKilometer, +dcid:wikidataId/Q1168,2003,dcid:Area_LandCover_Forest,55998,SquareKilometer, +dcid:wikidataId/Q1168,2005,dcid:Area_LandCover_Forest,55929,SquareKilometer, +dcid:wikidataId/Q46107,1987,dcid:Area_LandCover_Forest,238,SquareKilometer, +dcid:wikidataId/Q46107,1989,dcid:Area_LandCover_Forest,206,SquareKilometer, +dcid:wikidataId/Q46107,1991,dcid:Area_LandCover_Forest,206,SquareKilometer, +dcid:wikidataId/Q46107,1993,dcid:Area_LandCover_Forest,206,SquareKilometer, +dcid:wikidataId/Q46107,1995,dcid:Area_LandCover_Forest,204,SquareKilometer, +dcid:wikidataId/Q46107,1997,dcid:Area_LandCover_Forest,204,SquareKilometer, +dcid:wikidataId/Q46107,1999,dcid:Area_LandCover_Forest,202,SquareKilometer, +dcid:wikidataId/Q46107,2001,dcid:Area_LandCover_Forest,219,SquareKilometer, +dcid:wikidataId/Q46107,2003,dcid:Area_LandCover_Forest,225,SquareKilometer, +dcid:wikidataId/Q46107,2005,dcid:Area_LandCover_Forest,216,SquareKilometer, +dcid:wikidataId/Q66710,1995,dcid:Area_LandCover_Forest,3,SquareKilometer, +dcid:wikidataId/Q66710,1997,dcid:Area_LandCover_Forest,3,SquareKilometer, +dcid:wikidataId/Q66710,2001,dcid:Area_LandCover_Forest,6,SquareKilometer, +dcid:wikidataId/Q66710,2003,dcid:Area_LandCover_Forest,8,SquareKilometer, +dcid:wikidataId/Q66710,2005,dcid:Area_LandCover_Forest,6,SquareKilometer, +dcid:wikidataId/Q1353,1987,dcid:Area_LandCover_Forest,15,SquareKilometer, +dcid:wikidataId/Q1353,1989,dcid:Area_LandCover_Forest,22,SquareKilometer, +dcid:wikidataId/Q1353,1991,dcid:Area_LandCover_Forest,22,SquareKilometer, +dcid:wikidataId/Q1353,1993,dcid:Area_LandCover_Forest,22,SquareKilometer, +dcid:wikidataId/Q1353,1995,dcid:Area_LandCover_Forest,26,SquareKilometer, +dcid:wikidataId/Q1353,1997,dcid:Area_LandCover_Forest,26,SquareKilometer, +dcid:wikidataId/Q1353,1999,dcid:Area_LandCover_Forest,88,SquareKilometer, +dcid:wikidataId/Q1353,2001,dcid:Area_LandCover_Forest,111,SquareKilometer, +dcid:wikidataId/Q1353,2003,dcid:Area_LandCover_Forest,170,SquareKilometer, +dcid:wikidataId/Q1353,2005,dcid:Area_LandCover_Forest,177,SquareKilometer, +dcid:wikidataId/Q1171,1987,dcid:Area_LandCover_Forest,1240,SquareKilometer, +dcid:wikidataId/Q1171,1989,dcid:Area_LandCover_Forest,1255,SquareKilometer, +dcid:wikidataId/Q1171,1991,dcid:Area_LandCover_Forest,1255,SquareKilometer, +dcid:wikidataId/Q1171,1993,dcid:Area_LandCover_Forest,1250,SquareKilometer, +dcid:wikidataId/Q1171,1995,dcid:Area_LandCover_Forest,1247,SquareKilometer, +dcid:wikidataId/Q1171,1997,dcid:Area_LandCover_Forest,1252,SquareKilometer, +dcid:wikidataId/Q1171,1999,dcid:Area_LandCover_Forest,1251,SquareKilometer, +dcid:wikidataId/Q1171,2001,dcid:Area_LandCover_Forest,2095,SquareKilometer, +dcid:wikidataId/Q1171,2003,dcid:Area_LandCover_Forest,2156,SquareKilometer, +dcid:wikidataId/Q1171,2005,dcid:Area_LandCover_Forest,2156,SquareKilometer, +dcid:wikidataId/Q1061,1987,dcid:Area_LandCover_Forest,11991,SquareKilometer, +dcid:wikidataId/Q1061,1989,dcid:Area_LandCover_Forest,11921,SquareKilometer, +dcid:wikidataId/Q1061,1991,dcid:Area_LandCover_Forest,11907,SquareKilometer, +dcid:wikidataId/Q1061,1993,dcid:Area_LandCover_Forest,12044,SquareKilometer, +dcid:wikidataId/Q1061,1995,dcid:Area_LandCover_Forest,12320,SquareKilometer, +dcid:wikidataId/Q1061,1997,dcid:Area_LandCover_Forest,12578,SquareKilometer, +dcid:wikidataId/Q1061,1999,dcid:Area_LandCover_Forest,12965,SquareKilometer, +dcid:wikidataId/Q1061,2001,dcid:Area_LandCover_Forest,15152,SquareKilometer, +dcid:wikidataId/Q1061,2003,dcid:Area_LandCover_Forest,14946,SquareKilometer, +dcid:wikidataId/Q1061,2005,dcid:Area_LandCover_Forest,14604,SquareKilometer, +dcid:wikidataId/Q1174,1987,dcid:Area_LandCover_Forest,513,SquareKilometer, +dcid:wikidataId/Q1174,1989,dcid:Area_LandCover_Forest,513,SquareKilometer, +dcid:wikidataId/Q1174,1991,dcid:Area_LandCover_Forest,513,SquareKilometer, +dcid:wikidataId/Q1174,1993,dcid:Area_LandCover_Forest,513,SquareKilometer, +dcid:wikidataId/Q1174,1995,dcid:Area_LandCover_Forest,603,SquareKilometer, +dcid:wikidataId/Q1174,1997,dcid:Area_LandCover_Forest,604,SquareKilometer, +dcid:wikidataId/Q1174,1999,dcid:Area_LandCover_Forest,964,SquareKilometer, +dcid:wikidataId/Q1174,2001,dcid:Area_LandCover_Forest,1754,SquareKilometer, +dcid:wikidataId/Q1174,2003,dcid:Area_LandCover_Forest,1517,SquareKilometer, +dcid:wikidataId/Q1174,2005,dcid:Area_LandCover_Forest,1604,SquareKilometer, +dcid:wikidataId/Q1177,1987,dcid:Area_LandCover_Forest,12480,SquareKilometer, +dcid:wikidataId/Q1177,1989,dcid:Area_LandCover_Forest,12480,SquareKilometer, +dcid:wikidataId/Q1177,1991,dcid:Area_LandCover_Forest,12480,SquareKilometer, +dcid:wikidataId/Q1177,1993,dcid:Area_LandCover_Forest,12502,SquareKilometer, +dcid:wikidataId/Q1177,1995,dcid:Area_LandCover_Forest,12501,SquareKilometer, +dcid:wikidataId/Q1177,1997,dcid:Area_LandCover_Forest,12521,SquareKilometer, +dcid:wikidataId/Q1177,1999,dcid:Area_LandCover_Forest,13082,SquareKilometer, +dcid:wikidataId/Q1177,2001,dcid:Area_LandCover_Forest,14360,SquareKilometer, +dcid:wikidataId/Q1177,2003,dcid:Area_LandCover_Forest,14353,SquareKilometer, +dcid:wikidataId/Q1177,2005,dcid:Area_LandCover_Forest,14666,SquareKilometer, +dcid:wikidataId/Q66278313,1987,dcid:Area_LandCover_Forest,20905,SquareKilometer, +dcid:wikidataId/Q66278313,1989,dcid:Area_LandCover_Forest,20499,SquareKilometer, +dcid:wikidataId/Q66278313,1991,dcid:Area_LandCover_Forest,20499,SquareKilometer, +dcid:wikidataId/Q66278313,1993,dcid:Area_LandCover_Forest,20443,SquareKilometer, +dcid:wikidataId/Q66278313,1995,dcid:Area_LandCover_Forest,20433,SquareKilometer, +dcid:wikidataId/Q66278313,1997,dcid:Area_LandCover_Forest,20440,SquareKilometer, +dcid:wikidataId/Q66278313,1999,dcid:Area_LandCover_Forest,20441,SquareKilometer, +dcid:wikidataId/Q66278313,2001,dcid:Area_LandCover_Forest,21237,SquareKilometer, +dcid:wikidataId/Q66278313,2003,dcid:Area_LandCover_Forest,21267,SquareKilometer, +dcid:wikidataId/Q66278313,2005,dcid:Area_LandCover_Forest,22689,SquareKilometer, +dcid:wikidataId/Q1184,1997,dcid:Area_LandCover_Forest,21692,SquareKilometer, +dcid:wikidataId/Q1184,1999,dcid:Area_LandCover_Forest,21644,SquareKilometer, +dcid:wikidataId/Q1184,2001,dcid:Area_LandCover_Forest,22637,SquareKilometer, +dcid:wikidataId/Q1184,2003,dcid:Area_LandCover_Forest,22716,SquareKilometer, +dcid:wikidataId/Q1184,2005,dcid:Area_LandCover_Forest,22722,SquareKilometer, +dcid:wikidataId/Q1185,1987,dcid:Area_LandCover_Forest,32268,SquareKilometer, +dcid:wikidataId/Q1185,1989,dcid:Area_LandCover_Forest,32104,SquareKilometer, +dcid:wikidataId/Q1185,1991,dcid:Area_LandCover_Forest,32199,SquareKilometer, +dcid:wikidataId/Q1185,1993,dcid:Area_LandCover_Forest,32343,SquareKilometer, +dcid:wikidataId/Q1185,1995,dcid:Area_LandCover_Forest,32382,SquareKilometer, +dcid:wikidataId/Q1185,1997,dcid:Area_LandCover_Forest,32403,SquareKilometer, +dcid:wikidataId/Q1185,1999,dcid:Area_LandCover_Forest,32467,SquareKilometer, +dcid:wikidataId/Q1185,2001,dcid:Area_LandCover_Forest,36991,SquareKilometer, +dcid:wikidataId/Q1185,2003,dcid:Area_LandCover_Forest,36449,SquareKilometer, +dcid:wikidataId/Q1185,2005,dcid:Area_LandCover_Forest,36200,SquareKilometer, +dcid:wikidataId/Q1186,1987,dcid:Area_LandCover_Forest,10292,SquareKilometer, +dcid:wikidataId/Q1186,1989,dcid:Area_LandCover_Forest,10292,SquareKilometer, +dcid:wikidataId/Q1186,1991,dcid:Area_LandCover_Forest,10292,SquareKilometer, +dcid:wikidataId/Q1186,1993,dcid:Area_LandCover_Forest,10336,SquareKilometer, +dcid:wikidataId/Q1186,1995,dcid:Area_LandCover_Forest,10336,SquareKilometer, +dcid:wikidataId/Q1186,1997,dcid:Area_LandCover_Forest,10334,SquareKilometer, +dcid:wikidataId/Q1186,1999,dcid:Area_LandCover_Forest,10323,SquareKilometer, +dcid:wikidataId/Q1186,2001,dcid:Area_LandCover_Forest,15560,SquareKilometer, +dcid:wikidataId/Q1186,2003,dcid:Area_LandCover_Forest,15577,SquareKilometer, +dcid:wikidataId/Q1186,2005,dcid:Area_LandCover_Forest,17284,SquareKilometer, +dcid:wikidataId/Q26927,2001,dcid:Area_LandCover_Forest,27,SquareKilometer, +dcid:wikidataId/Q26927,2003,dcid:Area_LandCover_Forest,23,SquareKilometer, +dcid:wikidataId/Q26927,2005,dcid:Area_LandCover_Forest,26,SquareKilometer, +dcid:wikidataId/Q1188,1987,dcid:Area_LandCover_Forest,130099,SquareKilometer, +dcid:wikidataId/Q1188,1989,dcid:Area_LandCover_Forest,135541,SquareKilometer, +dcid:wikidataId/Q1188,1991,dcid:Area_LandCover_Forest,135541,SquareKilometer, +dcid:wikidataId/Q1188,1993,dcid:Area_LandCover_Forest,135396,SquareKilometer, +dcid:wikidataId/Q1188,1995,dcid:Area_LandCover_Forest,135164,SquareKilometer, +dcid:wikidataId/Q1188,1997,dcid:Area_LandCover_Forest,74760,SquareKilometer, +dcid:wikidataId/Q1188,1999,dcid:Area_LandCover_Forest,75137,SquareKilometer, +dcid:wikidataId/Q1188,2001,dcid:Area_LandCover_Forest,77265,SquareKilometer, +dcid:wikidataId/Q1188,2003,dcid:Area_LandCover_Forest,76429,SquareKilometer, +dcid:wikidataId/Q1188,2005,dcid:Area_LandCover_Forest,77739,SquareKilometer, +dcid:wikidataId/Q1191,1987,dcid:Area_LandCover_Forest,45616,SquareKilometer, +dcid:wikidataId/Q1191,1989,dcid:Area_LandCover_Forest,44044,SquareKilometer, +dcid:wikidataId/Q1191,1991,dcid:Area_LandCover_Forest,44044,SquareKilometer, +dcid:wikidataId/Q1191,1993,dcid:Area_LandCover_Forest,43859,SquareKilometer, +dcid:wikidataId/Q1191,1995,dcid:Area_LandCover_Forest,43843,SquareKilometer, +dcid:wikidataId/Q1191,1997,dcid:Area_LandCover_Forest,46143,SquareKilometer, +dcid:wikidataId/Q1191,1999,dcid:Area_LandCover_Forest,46672,SquareKilometer, +dcid:wikidataId/Q1191,2001,dcid:Area_LandCover_Forest,47482,SquareKilometer, +dcid:wikidataId/Q1191,2003,dcid:Area_LandCover_Forest,46865,SquareKilometer, +dcid:wikidataId/Q1191,2005,dcid:Area_LandCover_Forest,50661,SquareKilometer, +dcid:wikidataId/Q1193,1987,dcid:Area_LandCover_Forest,17475,SquareKilometer, +dcid:wikidataId/Q1193,1989,dcid:Area_LandCover_Forest,17685,SquareKilometer, +dcid:wikidataId/Q1193,1991,dcid:Area_LandCover_Forest,17685,SquareKilometer, +dcid:wikidataId/Q1193,1993,dcid:Area_LandCover_Forest,17621,SquareKilometer, +dcid:wikidataId/Q1193,1995,dcid:Area_LandCover_Forest,17558,SquareKilometer, +dcid:wikidataId/Q1193,1997,dcid:Area_LandCover_Forest,17418,SquareKilometer, +dcid:wikidataId/Q1193,1999,dcid:Area_LandCover_Forest,17384,SquareKilometer, +dcid:wikidataId/Q1193,2001,dcid:Area_LandCover_Forest,16926,SquareKilometer, +dcid:wikidataId/Q1193,2003,dcid:Area_LandCover_Forest,17219,SquareKilometer, +dcid:wikidataId/Q1193,2005,dcid:Area_LandCover_Forest,16952,SquareKilometer, +dcid:wikidataId/Q1195,1987,dcid:Area_LandCover_Forest,16466,SquareKilometer, +dcid:wikidataId/Q1195,1989,dcid:Area_LandCover_Forest,15645,SquareKilometer, +dcid:wikidataId/Q1195,1991,dcid:Area_LandCover_Forest,15875,SquareKilometer, +dcid:wikidataId/Q1195,1993,dcid:Area_LandCover_Forest,15769,SquareKilometer, +dcid:wikidataId/Q1195,1995,dcid:Area_LandCover_Forest,15714,SquareKilometer, +dcid:wikidataId/Q1195,1997,dcid:Area_LandCover_Forest,15657,SquareKilometer, +dcid:wikidataId/Q1195,1999,dcid:Area_LandCover_Forest,15633,SquareKilometer, +dcid:wikidataId/Q1195,2001,dcid:Area_LandCover_Forest,15584,SquareKilometer, +dcid:wikidataId/Q1195,2003,dcid:Area_LandCover_Forest,16839,SquareKilometer, +dcid:wikidataId/Q1195,2005,dcid:Area_LandCover_Forest,17205,SquareKilometer, +dcid:wikidataId/Q1502,1987,dcid:Area_LandCover_Forest,19084,SquareKilometer, +dcid:wikidataId/Q1502,1989,dcid:Area_LandCover_Forest,18170,SquareKilometer, +dcid:wikidataId/Q1502,1991,dcid:Area_LandCover_Forest,18853,SquareKilometer, +dcid:wikidataId/Q1502,1993,dcid:Area_LandCover_Forest,18697,SquareKilometer, +dcid:wikidataId/Q1502,1995,dcid:Area_LandCover_Forest,18576,SquareKilometer, +dcid:wikidataId/Q1502,1997,dcid:Area_LandCover_Forest,18775,SquareKilometer, +dcid:wikidataId/Q1502,1999,dcid:Area_LandCover_Forest,18338,SquareKilometer, +dcid:wikidataId/Q1502,2001,dcid:Area_LandCover_Forest,17494,SquareKilometer, +dcid:wikidataId/Q1502,2003,dcid:Area_LandCover_Forest,18430,SquareKilometer, +dcid:wikidataId/Q1502,2005,dcid:Area_LandCover_Forest,18600,SquareKilometer, +dcid:wikidataId/Q1599,1987,dcid:Area_LandCover_Forest,14394,SquareKilometer, +dcid:wikidataId/Q1599,1989,dcid:Area_LandCover_Forest,14399,SquareKilometer, +dcid:wikidataId/Q1599,1991,dcid:Area_LandCover_Forest,14321,SquareKilometer, +dcid:wikidataId/Q1599,1993,dcid:Area_LandCover_Forest,14348,SquareKilometer, +dcid:wikidataId/Q1599,1995,dcid:Area_LandCover_Forest,14291,SquareKilometer, +dcid:wikidataId/Q1599,1997,dcid:Area_LandCover_Forest,14221,SquareKilometer, +dcid:wikidataId/Q1599,1999,dcid:Area_LandCover_Forest,14164,SquareKilometer, +dcid:wikidataId/Q1599,2001,dcid:Area_LandCover_Forest,13345,SquareKilometer, +dcid:wikidataId/Q1599,2003,dcid:Area_LandCover_Forest,13609,SquareKilometer, +dcid:wikidataId/Q1599,2005,dcid:Area_LandCover_Forest,13665,SquareKilometer, +dcid:wikidataId/Q22048,1987,dcid:Area_LandCover_Forest,53253,SquareKilometer, +dcid:wikidataId/Q22048,1989,dcid:Area_LandCover_Forest,47227,SquareKilometer, +dcid:wikidataId/Q22048,1991,dcid:Area_LandCover_Forest,47205,SquareKilometer, +dcid:wikidataId/Q22048,1993,dcid:Area_LandCover_Forest,47145,SquareKilometer, +dcid:wikidataId/Q22048,1995,dcid:Area_LandCover_Forest,47107,SquareKilometer, +dcid:wikidataId/Q22048,1997,dcid:Area_LandCover_Forest,46941,SquareKilometer, +dcid:wikidataId/Q22048,1999,dcid:Area_LandCover_Forest,47033,SquareKilometer, +dcid:wikidataId/Q22048,2001,dcid:Area_LandCover_Forest,48838,SquareKilometer, +dcid:wikidataId/Q22048,2003,dcid:Area_LandCover_Forest,48366,SquareKilometer, +dcid:wikidataId/Q22048,2005,dcid:Area_LandCover_Forest,48755,SquareKilometer, +dcid:wikidataId/Q66743,2001,dcid:Area_LandCover_Forest,36,SquareKilometer, +dcid:wikidataId/Q66743,2003,dcid:Area_LandCover_Forest,40,SquareKilometer, +dcid:wikidataId/Q66743,2005,dcid:Area_LandCover_Forest,42,SquareKilometer, +dcid:wikidataId/Q22424,1987,dcid:Area_LandCover_Forest,943,SquareKilometer, +dcid:wikidataId/Q22424,1989,dcid:Area_LandCover_Forest,1338,SquareKilometer, +dcid:wikidataId/Q22424,1991,dcid:Area_LandCover_Forest,1343,SquareKilometer, +dcid:wikidataId/Q22424,1993,dcid:Area_LandCover_Forest,1343,SquareKilometer, +dcid:wikidataId/Q22424,1995,dcid:Area_LandCover_Forest,1342,SquareKilometer, +dcid:wikidataId/Q22424,1997,dcid:Area_LandCover_Forest,1387,SquareKilometer, +dcid:wikidataId/Q22424,1999,dcid:Area_LandCover_Forest,1412,SquareKilometer, +dcid:wikidataId/Q22424,2001,dcid:Area_LandCover_Forest,2432,SquareKilometer, +dcid:wikidataId/Q22424,2003,dcid:Area_LandCover_Forest,1580,SquareKilometer, +dcid:wikidataId/Q22424,2005,dcid:Area_LandCover_Forest,1660,SquareKilometer, +dcid:wikidataId/Q1437,1987,dcid:Area_LandCover_Forest,12758,SquareKilometer, +dcid:wikidataId/Q1437,1989,dcid:Area_LandCover_Forest,12884,SquareKilometer, +dcid:wikidataId/Q1437,1991,dcid:Area_LandCover_Forest,12889,SquareKilometer, +dcid:wikidataId/Q1437,1993,dcid:Area_LandCover_Forest,13099,SquareKilometer, +dcid:wikidataId/Q1437,1995,dcid:Area_LandCover_Forest,13280,SquareKilometer, +dcid:wikidataId/Q1437,1997,dcid:Area_LandCover_Forest,13353,SquareKilometer, +dcid:wikidataId/Q1437,1999,dcid:Area_LandCover_Forest,13871,SquareKilometer, +dcid:wikidataId/Q1437,2001,dcid:Area_LandCover_Forest,16367,SquareKilometer, +dcid:wikidataId/Q1437,2003,dcid:Area_LandCover_Forest,15826,SquareKilometer, +dcid:wikidataId/Q1437,2005,dcid:Area_LandCover_Forest,16012,SquareKilometer, +dcid:wikidataId/Q1505,1987,dcid:Area_LandCover_Forest,2756,SquareKilometer, +dcid:wikidataId/Q1505,1989,dcid:Area_LandCover_Forest,3041,SquareKilometer, +dcid:wikidataId/Q1505,1991,dcid:Area_LandCover_Forest,3041,SquareKilometer, +dcid:wikidataId/Q1505,1993,dcid:Area_LandCover_Forest,3119,SquareKilometer, +dcid:wikidataId/Q1505,1995,dcid:Area_LandCover_Forest,3127,SquareKilometer, +dcid:wikidataId/Q1505,1997,dcid:Area_LandCover_Forest,3129,SquareKilometer, +dcid:wikidataId/Q1505,1999,dcid:Area_LandCover_Forest,3118,SquareKilometer, +dcid:wikidataId/Q1505,2001,dcid:Area_LandCover_Forest,3193,SquareKilometer, +dcid:wikidataId/Q1505,2003,dcid:Area_LandCover_Forest,3262,SquareKilometer, +dcid:wikidataId/Q1505,2005,dcid:Area_LandCover_Forest,3357,SquareKilometer, +dcid:wikidataId/Q1445,1987,dcid:Area_LandCover_Forest,17472,SquareKilometer, +dcid:wikidataId/Q1445,1989,dcid:Area_LandCover_Forest,16992,SquareKilometer, +dcid:wikidataId/Q1445,1991,dcid:Area_LandCover_Forest,16992,SquareKilometer, +dcid:wikidataId/Q1445,1993,dcid:Area_LandCover_Forest,17005,SquareKilometer, +dcid:wikidataId/Q1445,1995,dcid:Area_LandCover_Forest,17045,SquareKilometer, +dcid:wikidataId/Q1445,1997,dcid:Area_LandCover_Forest,17064,SquareKilometer, +dcid:wikidataId/Q1445,1999,dcid:Area_LandCover_Forest,17078,SquareKilometer, +dcid:wikidataId/Q1445,2001,dcid:Area_LandCover_Forest,21482,SquareKilometer, +dcid:wikidataId/Q1445,2003,dcid:Area_LandCover_Forest,22643,SquareKilometer, +dcid:wikidataId/Q1445,2005,dcid:Area_LandCover_Forest,23314,SquareKilometer, +dcid:wikidataId/Q1363,1987,dcid:Area_LandCover_Forest,5953,SquareKilometer, +dcid:wikidataId/Q1363,1989,dcid:Area_LandCover_Forest,5535,SquareKilometer, +dcid:wikidataId/Q1363,1991,dcid:Area_LandCover_Forest,5535,SquareKilometer, +dcid:wikidataId/Q1363,1993,dcid:Area_LandCover_Forest,5538,SquareKilometer, +dcid:wikidataId/Q1363,1995,dcid:Area_LandCover_Forest,5538,SquareKilometer, +dcid:wikidataId/Q1363,1997,dcid:Area_LandCover_Forest,5546,SquareKilometer, +dcid:wikidataId/Q1363,1999,dcid:Area_LandCover_Forest,5745,SquareKilometer, +dcid:wikidataId/Q1363,2001,dcid:Area_LandCover_Forest,7065,SquareKilometer, +dcid:wikidataId/Q1363,2003,dcid:Area_LandCover_Forest,8093,SquareKilometer, +dcid:wikidataId/Q1363,2005,dcid:Area_LandCover_Forest,8173,SquareKilometer, +dcid:wikidataId/Q1498,1987,dcid:Area_LandCover_Forest,31226,SquareKilometer, +dcid:wikidataId/Q1498,1989,dcid:Area_LandCover_Forest,33627,SquareKilometer, +dcid:wikidataId/Q1498,1991,dcid:Area_LandCover_Forest,33609,SquareKilometer, +dcid:wikidataId/Q1498,1993,dcid:Area_LandCover_Forest,33961,SquareKilometer, +dcid:wikidataId/Q1498,1995,dcid:Area_LandCover_Forest,33986,SquareKilometer, +dcid:wikidataId/Q1498,1997,dcid:Area_LandCover_Forest,10751,SquareKilometer, +dcid:wikidataId/Q1498,1999,dcid:Area_LandCover_Forest,10756,SquareKilometer, +dcid:wikidataId/Q1498,2001,dcid:Area_LandCover_Forest,13746,SquareKilometer, +dcid:wikidataId/Q1498,2003,dcid:Area_LandCover_Forest,14118,SquareKilometer, +dcid:wikidataId/Q1498,2005,dcid:Area_LandCover_Forest,14346,SquareKilometer, +dcid:wikidataId/Q1499,1997,dcid:Area_LandCover_Forest,23243,SquareKilometer, +dcid:wikidataId/Q1499,1999,dcid:Area_LandCover_Forest,23260,SquareKilometer, +dcid:wikidataId/Q1499,2001,dcid:Area_LandCover_Forest,23938,SquareKilometer, +dcid:wikidataId/Q1499,2003,dcid:Area_LandCover_Forest,24465,SquareKilometer, +dcid:wikidataId/Q1499,2005,dcid:Area_LandCover_Forest,24493,SquareKilometer, +dcid:wikidataId/Q1356,1987,dcid:Area_LandCover_Forest,8432,SquareKilometer, +dcid:wikidataId/Q1356,1989,dcid:Area_LandCover_Forest,8015,SquareKilometer, +dcid:wikidataId/Q1356,1991,dcid:Area_LandCover_Forest,8015,SquareKilometer, +dcid:wikidataId/Q1356,1993,dcid:Area_LandCover_Forest,8186,SquareKilometer, +dcid:wikidataId/Q1356,1995,dcid:Area_LandCover_Forest,8276,SquareKilometer, +dcid:wikidataId/Q1356,1997,dcid:Area_LandCover_Forest,8349,SquareKilometer, +dcid:wikidataId/Q1356,1999,dcid:Area_LandCover_Forest,8362,SquareKilometer, +dcid:wikidataId/Q1356,2001,dcid:Area_LandCover_Forest,10693,SquareKilometer, +dcid:wikidataId/Q1356,2003,dcid:Area_LandCover_Forest,12343,SquareKilometer, +dcid:wikidataId/Q1356,2005,dcid:Area_LandCover_Forest,12970,SquareKilometer, +dcid:country/IND,1987,dcid:Area_LandCover_Forest,640819,SquareKilometer, +dcid:country/IND,1989,dcid:Area_LandCover_Forest,638804,SquareKilometer, +dcid:country/IND,1991,dcid:Area_LandCover_Forest,639364,SquareKilometer, +dcid:country/IND,1993,dcid:Area_LandCover_Forest,639386,SquareKilometer, +dcid:country/IND,1995,dcid:Area_LandCover_Forest,638879,SquareKilometer, +dcid:country/IND,1997,dcid:Area_LandCover_Forest,633397,SquareKilometer, +dcid:country/IND,1999,dcid:Area_LandCover_Forest,637293,SquareKilometer, +dcid:country/IND,2001,dcid:Area_LandCover_Forest,675538,SquareKilometer, +dcid:country/IND,2003,dcid:Area_LandCover_Forest,678333,SquareKilometer, +dcid:country/IND,2005,dcid:Area_LandCover_Forest,690171,SquareKilometer, +dcid:wikidataId/Q40888,2007,dcid:Area_LandCover_Forest,6662,SquareKilometer, +dcid:wikidataId/Q40888,2009,dcid:Area_LandCover_Forest,6662,SquareKilometer, +dcid:wikidataId/Q40888,2011,dcid:Area_LandCover_Forest,6724,SquareKilometer, +dcid:wikidataId/Q40888,2013,dcid:Area_LandCover_Forest,6711,SquareKilometer, +dcid:wikidataId/Q40888,2015,dcid:Area_LandCover_Forest,6751,SquareKilometer, +dcid:wikidataId/Q40888,2017,dcid:Area_LandCover_Forest,6742,SquareKilometer, +dcid:wikidataId/Q40888,2019,dcid:Area_LandCover_Forest,6743,SquareKilometer, +dcid:wikidataId/Q40888,2021,dcid:Area_LandCover_Forest,6737,SquareKilometer, +dcid:wikidataId/Q40888,2023,dcid:Area_LandCover_Forest,6733,SquareKilometer, +dcid:wikidataId/Q1159,2007,dcid:Area_LandCover_Forest,45102,SquareKilometer, +dcid:wikidataId/Q1159,2009,dcid:Area_LandCover_Forest,46670,SquareKilometer, +dcid:wikidataId/Q1159,2011,dcid:Area_LandCover_Forest,46389,SquareKilometer, +dcid:wikidataId/Q1159,2013,dcid:Area_LandCover_Forest,26044,SquareKilometer, +dcid:wikidataId/Q1159,2015,dcid:Area_LandCover_Forest,26006,SquareKilometer, +dcid:wikidataId/Q1159,2017,dcid:Area_LandCover_Forest,28147,SquareKilometer, +dcid:wikidataId/Q1159,2019,dcid:Area_LandCover_Forest,29137,SquareKilometer, +dcid:wikidataId/Q1159,2021,dcid:Area_LandCover_Forest,30224,SquareKilometer, +dcid:wikidataId/Q1159,2023,dcid:Area_LandCover_Forest,30085,SquareKilometer, +dcid:wikidataId/Q1162,2007,dcid:Area_LandCover_Forest,67353,SquareKilometer, +dcid:wikidataId/Q1162,2009,dcid:Area_LandCover_Forest,67484,SquareKilometer, +dcid:wikidataId/Q1162,2011,dcid:Area_LandCover_Forest,67410,SquareKilometer, +dcid:wikidataId/Q1162,2013,dcid:Area_LandCover_Forest,66966,SquareKilometer, +dcid:wikidataId/Q1162,2015,dcid:Area_LandCover_Forest,67154,SquareKilometer, +dcid:wikidataId/Q1162,2017,dcid:Area_LandCover_Forest,66964,SquareKilometer, +dcid:wikidataId/Q1162,2019,dcid:Area_LandCover_Forest,66688,SquareKilometer, +dcid:wikidataId/Q1162,2021,dcid:Area_LandCover_Forest,65973,SquareKilometer, +dcid:wikidataId/Q1162,2023,dcid:Area_LandCover_Forest,65882,SquareKilometer, +dcid:wikidataId/Q1164,2007,dcid:Area_LandCover_Forest,27692,SquareKilometer, +dcid:wikidataId/Q1164,2009,dcid:Area_LandCover_Forest,27692,SquareKilometer, +dcid:wikidataId/Q1164,2011,dcid:Area_LandCover_Forest,27673,SquareKilometer, +dcid:wikidataId/Q1164,2013,dcid:Area_LandCover_Forest,27565,SquareKilometer, +dcid:wikidataId/Q1164,2015,dcid:Area_LandCover_Forest,27538,SquareKilometer, +dcid:wikidataId/Q1164,2017,dcid:Area_LandCover_Forest,28105,SquareKilometer, +dcid:wikidataId/Q1164,2019,dcid:Area_LandCover_Forest,28327,SquareKilometer, +dcid:wikidataId/Q1164,2021,dcid:Area_LandCover_Forest,28325,SquareKilometer, +dcid:wikidataId/Q1164,2023,dcid:Area_LandCover_Forest,28314,SquareKilometer, +dcid:wikidataId/Q1165,2007,dcid:Area_LandCover_Forest,6804,SquareKilometer, +dcid:wikidataId/Q1165,2009,dcid:Area_LandCover_Forest,6804,SquareKilometer, +dcid:wikidataId/Q1165,2011,dcid:Area_LandCover_Forest,6845,SquareKilometer, +dcid:wikidataId/Q1165,2013,dcid:Area_LandCover_Forest,7284,SquareKilometer, +dcid:wikidataId/Q1165,2015,dcid:Area_LandCover_Forest,7254,SquareKilometer, +dcid:wikidataId/Q1165,2017,dcid:Area_LandCover_Forest,7299,SquareKilometer, +dcid:wikidataId/Q1165,2019,dcid:Area_LandCover_Forest,7306,SquareKilometer, +dcid:wikidataId/Q1165,2021,dcid:Area_LandCover_Forest,7403,SquareKilometer, +dcid:wikidataId/Q1165,2023,dcid:Area_LandCover_Forest,7532,SquareKilometer, +dcid:wikidataId/Q43433,2007,dcid:Area_LandCover_Forest,17,SquareKilometer, +dcid:wikidataId/Q43433,2009,dcid:Area_LandCover_Forest,17,SquareKilometer, +dcid:wikidataId/Q43433,2011,dcid:Area_LandCover_Forest,17,SquareKilometer, +dcid:wikidataId/Q43433,2013,dcid:Area_LandCover_Forest,17,SquareKilometer, +dcid:wikidataId/Q43433,2015,dcid:Area_LandCover_Forest,22,SquareKilometer, +dcid:wikidataId/Q43433,2017,dcid:Area_LandCover_Forest,22,SquareKilometer, +dcid:wikidataId/Q43433,2019,dcid:Area_LandCover_Forest,22,SquareKilometer, +dcid:wikidataId/Q43433,2021,dcid:Area_LandCover_Forest,24,SquareKilometer, +dcid:wikidataId/Q43433,2023,dcid:Area_LandCover_Forest,25,SquareKilometer, +dcid:wikidataId/Q1168,2007,dcid:Area_LandCover_Forest,55870,SquareKilometer, +dcid:wikidataId/Q1168,2009,dcid:Area_LandCover_Forest,55678,SquareKilometer, +dcid:wikidataId/Q1168,2011,dcid:Area_LandCover_Forest,55674,SquareKilometer, +dcid:wikidataId/Q1168,2013,dcid:Area_LandCover_Forest,55607,SquareKilometer, +dcid:wikidataId/Q1168,2015,dcid:Area_LandCover_Forest,55559,SquareKilometer, +dcid:wikidataId/Q1168,2017,dcid:Area_LandCover_Forest,55547,SquareKilometer, +dcid:wikidataId/Q1168,2019,dcid:Area_LandCover_Forest,55611,SquareKilometer, +dcid:wikidataId/Q1168,2021,dcid:Area_LandCover_Forest,55831,SquareKilometer, +dcid:wikidataId/Q1168,2023,dcid:Area_LandCover_Forest,55812,SquareKilometer, +dcid:wikidataId/Q46107,2007,dcid:Area_LandCover_Forest,211,SquareKilometer, +dcid:wikidataId/Q46107,2009,dcid:Area_LandCover_Forest,211,SquareKilometer, +dcid:wikidataId/Q46107,2011,dcid:Area_LandCover_Forest,211,SquareKilometer, +dcid:wikidataId/Q46107,2013,dcid:Area_LandCover_Forest,220,SquareKilometer, +dcid:wikidataId/Q46107,2015,dcid:Area_LandCover_Forest,206,SquareKilometer, +dcid:wikidataId/Q46107,2017,dcid:Area_LandCover_Forest,207,SquareKilometer, +dcid:wikidataId/Q46107,2019,dcid:Area_LandCover_Forest,207,SquareKilometer, +dcid:wikidataId/Q46107,2021,dcid:Area_LandCover_Forest,226,SquareKilometer, +dcid:wikidataId/Q46107,2023,dcid:Area_LandCover_Forest,226,SquareKilometer, +dcid:wikidataId/Q66710,2007,dcid:Area_LandCover_Forest,6,SquareKilometer, +dcid:wikidataId/Q66710,2009,dcid:Area_LandCover_Forest,6,SquareKilometer, +dcid:wikidataId/Q66710,2011,dcid:Area_LandCover_Forest,6,SquareKilometer, +dcid:wikidataId/Q66710,2015,dcid:Area_LandCover_Forest,20,SquareKilometer, +dcid:wikidataId/Q66710,2017,dcid:Area_LandCover_Forest,20,SquareKilometer, +dcid:wikidataId/Q66710,2019,dcid:Area_LandCover_Forest,20,SquareKilometer, +dcid:wikidataId/Q1353,2007,dcid:Area_LandCover_Forest,177,SquareKilometer, +dcid:wikidataId/Q1353,2009,dcid:Area_LandCover_Forest,177,SquareKilometer, +dcid:wikidataId/Q1353,2011,dcid:Area_LandCover_Forest,176,SquareKilometer, +dcid:wikidataId/Q1353,2013,dcid:Area_LandCover_Forest,180,SquareKilometer, +dcid:wikidataId/Q1353,2015,dcid:Area_LandCover_Forest,189,SquareKilometer, +dcid:wikidataId/Q1353,2017,dcid:Area_LandCover_Forest,192,SquareKilometer, +dcid:wikidataId/Q1353,2019,dcid:Area_LandCover_Forest,195,SquareKilometer, +dcid:wikidataId/Q1353,2021,dcid:Area_LandCover_Forest,195,SquareKilometer, +dcid:wikidataId/Q1353,2023,dcid:Area_LandCover_Forest,195,SquareKilometer, +dcid:wikidataId/Q1171,2007,dcid:Area_LandCover_Forest,2151,SquareKilometer, +dcid:wikidataId/Q1171,2009,dcid:Area_LandCover_Forest,2212,SquareKilometer, +dcid:wikidataId/Q1171,2011,dcid:Area_LandCover_Forest,2219,SquareKilometer, +dcid:wikidataId/Q1171,2013,dcid:Area_LandCover_Forest,2219,SquareKilometer, +dcid:wikidataId/Q1171,2015,dcid:Area_LandCover_Forest,2210,SquareKilometer, +dcid:wikidataId/Q1171,2017,dcid:Area_LandCover_Forest,2229,SquareKilometer, +dcid:wikidataId/Q1171,2019,dcid:Area_LandCover_Forest,2237,SquareKilometer, +dcid:wikidataId/Q1171,2021,dcid:Area_LandCover_Forest,2267,SquareKilometer, +dcid:wikidataId/Q1171,2023,dcid:Area_LandCover_Forest,2266,SquareKilometer, +dcid:wikidataId/Q1061,2007,dcid:Area_LandCover_Forest,14620,SquareKilometer, +dcid:wikidataId/Q1061,2009,dcid:Area_LandCover_Forest,14620,SquareKilometer, +dcid:wikidataId/Q1061,2011,dcid:Area_LandCover_Forest,14619,SquareKilometer, +dcid:wikidataId/Q1061,2013,dcid:Area_LandCover_Forest,14508,SquareKilometer, +dcid:wikidataId/Q1061,2015,dcid:Area_LandCover_Forest,14710,SquareKilometer, +dcid:wikidataId/Q1061,2017,dcid:Area_LandCover_Forest,14757,SquareKilometer, +dcid:wikidataId/Q1061,2019,dcid:Area_LandCover_Forest,14857,SquareKilometer, +dcid:wikidataId/Q1061,2021,dcid:Area_LandCover_Forest,14837,SquareKilometer, +dcid:wikidataId/Q1061,2023,dcid:Area_LandCover_Forest,15017,SquareKilometer, +dcid:wikidataId/Q1174,2007,dcid:Area_LandCover_Forest,1594,SquareKilometer, +dcid:wikidataId/Q1174,2009,dcid:Area_LandCover_Forest,1594,SquareKilometer, +dcid:wikidataId/Q1174,2011,dcid:Area_LandCover_Forest,1608,SquareKilometer, +dcid:wikidataId/Q1174,2013,dcid:Area_LandCover_Forest,1583,SquareKilometer, +dcid:wikidataId/Q1174,2015,dcid:Area_LandCover_Forest,1580,SquareKilometer, +dcid:wikidataId/Q1174,2017,dcid:Area_LandCover_Forest,1588,SquareKilometer, +dcid:wikidataId/Q1174,2019,dcid:Area_LandCover_Forest,1602,SquareKilometer, +dcid:wikidataId/Q1174,2021,dcid:Area_LandCover_Forest,1628,SquareKilometer, +dcid:wikidataId/Q1174,2023,dcid:Area_LandCover_Forest,1614,SquareKilometer, +dcid:wikidataId/Q1177,2007,dcid:Area_LandCover_Forest,14668,SquareKilometer, +dcid:wikidataId/Q1177,2009,dcid:Area_LandCover_Forest,14668,SquareKilometer, +dcid:wikidataId/Q1177,2011,dcid:Area_LandCover_Forest,14679,SquareKilometer, +dcid:wikidataId/Q1177,2013,dcid:Area_LandCover_Forest,14683,SquareKilometer, +dcid:wikidataId/Q1177,2015,dcid:Area_LandCover_Forest,14707,SquareKilometer, +dcid:wikidataId/Q1177,2017,dcid:Area_LandCover_Forest,15100,SquareKilometer, +dcid:wikidataId/Q1177,2019,dcid:Area_LandCover_Forest,15434,SquareKilometer, +dcid:wikidataId/Q1177,2021,dcid:Area_LandCover_Forest,15526,SquareKilometer, +dcid:wikidataId/Q1177,2023,dcid:Area_LandCover_Forest,15580,SquareKilometer, +dcid:wikidataId/Q66278313,2007,dcid:Area_LandCover_Forest,22686,SquareKilometer, +dcid:wikidataId/Q66278313,2009,dcid:Area_LandCover_Forest,22537,SquareKilometer, +dcid:wikidataId/Q66278313,2011,dcid:Area_LandCover_Forest,22539,SquareKilometer, +dcid:wikidataId/Q66278313,2013,dcid:Area_LandCover_Forest,20948,SquareKilometer, +dcid:wikidataId/Q66278313,2015,dcid:Area_LandCover_Forest,22988,SquareKilometer, +dcid:wikidataId/Q66278313,2017,dcid:Area_LandCover_Forest,23241,SquareKilometer, +dcid:wikidataId/Q66278313,2019,dcid:Area_LandCover_Forest,23612,SquareKilometer, +dcid:wikidataId/Q66278313,2021,dcid:Area_LandCover_Forest,21263,SquareKilometer, +dcid:wikidataId/Q66278313,2023,dcid:Area_LandCover_Forest,21346,SquareKilometer, +dcid:wikidataId/Q1184,2007,dcid:Area_LandCover_Forest,22894,SquareKilometer, +dcid:wikidataId/Q1184,2009,dcid:Area_LandCover_Forest,22894,SquareKilometer, +dcid:wikidataId/Q1184,2011,dcid:Area_LandCover_Forest,22977,SquareKilometer, +dcid:wikidataId/Q1184,2013,dcid:Area_LandCover_Forest,23815,SquareKilometer, +dcid:wikidataId/Q1184,2015,dcid:Area_LandCover_Forest,23524,SquareKilometer, +dcid:wikidataId/Q1184,2017,dcid:Area_LandCover_Forest,23553,SquareKilometer, +dcid:wikidataId/Q1184,2019,dcid:Area_LandCover_Forest,23611,SquareKilometer, +dcid:wikidataId/Q1184,2021,dcid:Area_LandCover_Forest,23707,SquareKilometer, +dcid:wikidataId/Q1184,2023,dcid:Area_LandCover_Forest,23766,SquareKilometer, +dcid:wikidataId/Q1185,2007,dcid:Area_LandCover_Forest,36190,SquareKilometer, +dcid:wikidataId/Q1185,2009,dcid:Area_LandCover_Forest,36190,SquareKilometer, +dcid:wikidataId/Q1185,2011,dcid:Area_LandCover_Forest,36194,SquareKilometer, +dcid:wikidataId/Q1185,2013,dcid:Area_LandCover_Forest,36171,SquareKilometer, +dcid:wikidataId/Q1185,2015,dcid:Area_LandCover_Forest,36449,SquareKilometer, +dcid:wikidataId/Q1185,2017,dcid:Area_LandCover_Forest,37550,SquareKilometer, +dcid:wikidataId/Q1185,2019,dcid:Area_LandCover_Forest,38575,SquareKilometer, +dcid:wikidataId/Q1185,2021,dcid:Area_LandCover_Forest,39107,SquareKilometer, +dcid:wikidataId/Q1185,2023,dcid:Area_LandCover_Forest,39254,SquareKilometer, +dcid:wikidataId/Q1186,2007,dcid:Area_LandCover_Forest,17324,SquareKilometer, +dcid:wikidataId/Q1186,2009,dcid:Area_LandCover_Forest,17324,SquareKilometer, +dcid:wikidataId/Q1186,2011,dcid:Area_LandCover_Forest,17300,SquareKilometer, +dcid:wikidataId/Q1186,2013,dcid:Area_LandCover_Forest,18384,SquareKilometer, +dcid:wikidataId/Q1186,2015,dcid:Area_LandCover_Forest,19278,SquareKilometer, +dcid:wikidataId/Q1186,2017,dcid:Area_LandCover_Forest,20321,SquareKilometer, +dcid:wikidataId/Q1186,2019,dcid:Area_LandCover_Forest,21144,SquareKilometer, +dcid:wikidataId/Q1186,2021,dcid:Area_LandCover_Forest,21926,SquareKilometer, +dcid:wikidataId/Q1186,2023,dcid:Area_LandCover_Forest,22059,SquareKilometer, +dcid:wikidataId/Q26927,2007,dcid:Area_LandCover_Forest,26,SquareKilometer, +dcid:wikidataId/Q26927,2009,dcid:Area_LandCover_Forest,26,SquareKilometer, +dcid:wikidataId/Q26927,2011,dcid:Area_LandCover_Forest,27,SquareKilometer, +dcid:wikidataId/Q26927,2013,dcid:Area_LandCover_Forest,27,SquareKilometer, +dcid:wikidataId/Q26927,2015,dcid:Area_LandCover_Forest,27,SquareKilometer, +dcid:wikidataId/Q26927,2017,dcid:Area_LandCover_Forest,27,SquareKilometer, +dcid:wikidataId/Q26927,2019,dcid:Area_LandCover_Forest,27,SquareKilometer, +dcid:wikidataId/Q26927,2021,dcid:Area_LandCover_Forest,27,SquareKilometer, +dcid:wikidataId/Q26927,2023,dcid:Area_LandCover_Forest,27,SquareKilometer, +dcid:wikidataId/Q1188,2007,dcid:Area_LandCover_Forest,77700,SquareKilometer, +dcid:wikidataId/Q1188,2009,dcid:Area_LandCover_Forest,77700,SquareKilometer, +dcid:wikidataId/Q1188,2011,dcid:Area_LandCover_Forest,77700,SquareKilometer, +dcid:wikidataId/Q1188,2013,dcid:Area_LandCover_Forest,77522,SquareKilometer, +dcid:wikidataId/Q1188,2015,dcid:Area_LandCover_Forest,77426,SquareKilometer, +dcid:wikidataId/Q1188,2017,dcid:Area_LandCover_Forest,77414,SquareKilometer, +dcid:wikidataId/Q1188,2019,dcid:Area_LandCover_Forest,77482,SquareKilometer, +dcid:wikidataId/Q1188,2021,dcid:Area_LandCover_Forest,77445,SquareKilometer, +dcid:wikidataId/Q1188,2023,dcid:Area_LandCover_Forest,77073,SquareKilometer, +dcid:wikidataId/Q1191,2007,dcid:Area_LandCover_Forest,50650,SquareKilometer, +dcid:wikidataId/Q1191,2009,dcid:Area_LandCover_Forest,50650,SquareKilometer, +dcid:wikidataId/Q1191,2011,dcid:Area_LandCover_Forest,50646,SquareKilometer, +dcid:wikidataId/Q1191,2013,dcid:Area_LandCover_Forest,50632,SquareKilometer, +dcid:wikidataId/Q1191,2015,dcid:Area_LandCover_Forest,50699,SquareKilometer, +dcid:wikidataId/Q1191,2017,dcid:Area_LandCover_Forest,50682,SquareKilometer, +dcid:wikidataId/Q1191,2019,dcid:Area_LandCover_Forest,50778,SquareKilometer, +dcid:wikidataId/Q1191,2021,dcid:Area_LandCover_Forest,50913,SquareKilometer, +dcid:wikidataId/Q1191,2023,dcid:Area_LandCover_Forest,50859,SquareKilometer, +dcid:wikidataId/Q1193,2007,dcid:Area_LandCover_Forest,17280,SquareKilometer, +dcid:wikidataId/Q1193,2009,dcid:Area_LandCover_Forest,17280,SquareKilometer, +dcid:wikidataId/Q1193,2011,dcid:Area_LandCover_Forest,17090,SquareKilometer, +dcid:wikidataId/Q1193,2013,dcid:Area_LandCover_Forest,16961,SquareKilometer, +dcid:wikidataId/Q1193,2015,dcid:Area_LandCover_Forest,17083,SquareKilometer, +dcid:wikidataId/Q1193,2017,dcid:Area_LandCover_Forest,17346,SquareKilometer, +dcid:wikidataId/Q1193,2019,dcid:Area_LandCover_Forest,16847,SquareKilometer, +dcid:wikidataId/Q1193,2021,dcid:Area_LandCover_Forest,16640,SquareKilometer, +dcid:wikidataId/Q1193,2023,dcid:Area_LandCover_Forest,16585,SquareKilometer, +dcid:wikidataId/Q1195,2007,dcid:Area_LandCover_Forest,17321,SquareKilometer, +dcid:wikidataId/Q1195,2009,dcid:Area_LandCover_Forest,17321,SquareKilometer, +dcid:wikidataId/Q1195,2011,dcid:Area_LandCover_Forest,17275,SquareKilometer, +dcid:wikidataId/Q1195,2013,dcid:Area_LandCover_Forest,17241,SquareKilometer, +dcid:wikidataId/Q1195,2015,dcid:Area_LandCover_Forest,17262,SquareKilometer, +dcid:wikidataId/Q1195,2017,dcid:Area_LandCover_Forest,17146,SquareKilometer, +dcid:wikidataId/Q1195,2019,dcid:Area_LandCover_Forest,17119,SquareKilometer, +dcid:wikidataId/Q1195,2021,dcid:Area_LandCover_Forest,16997,SquareKilometer, +dcid:wikidataId/Q1195,2023,dcid:Area_LandCover_Forest,16967,SquareKilometer, +dcid:wikidataId/Q1502,2007,dcid:Area_LandCover_Forest,19240,SquareKilometer, +dcid:wikidataId/Q1502,2009,dcid:Area_LandCover_Forest,19183,SquareKilometer, +dcid:wikidataId/Q1502,2011,dcid:Area_LandCover_Forest,19117,SquareKilometer, +dcid:wikidataId/Q1502,2013,dcid:Area_LandCover_Forest,18978,SquareKilometer, +dcid:wikidataId/Q1502,2015,dcid:Area_LandCover_Forest,18717,SquareKilometer, +dcid:wikidataId/Q1502,2017,dcid:Area_LandCover_Forest,18186,SquareKilometer, +dcid:wikidataId/Q1502,2019,dcid:Area_LandCover_Forest,18006,SquareKilometer, +dcid:wikidataId/Q1502,2021,dcid:Area_LandCover_Forest,17749,SquareKilometer, +dcid:wikidataId/Q1502,2023,dcid:Area_LandCover_Forest,17990,SquareKilometer, +dcid:wikidataId/Q1599,2007,dcid:Area_LandCover_Forest,13464,SquareKilometer, +dcid:wikidataId/Q1599,2009,dcid:Area_LandCover_Forest,13464,SquareKilometer, +dcid:wikidataId/Q1599,2011,dcid:Area_LandCover_Forest,13318,SquareKilometer, +dcid:wikidataId/Q1599,2013,dcid:Area_LandCover_Forest,13017,SquareKilometer, +dcid:wikidataId/Q1599,2015,dcid:Area_LandCover_Forest,12939,SquareKilometer, +dcid:wikidataId/Q1599,2017,dcid:Area_LandCover_Forest,12489,SquareKilometer, +dcid:wikidataId/Q1599,2019,dcid:Area_LandCover_Forest,12486,SquareKilometer, +dcid:wikidataId/Q1599,2021,dcid:Area_LandCover_Forest,12274,SquareKilometer, +dcid:wikidataId/Q1599,2023,dcid:Area_LandCover_Forest,12222,SquareKilometer, +dcid:wikidataId/Q22048,2007,dcid:Area_LandCover_Forest,48855,SquareKilometer, +dcid:wikidataId/Q22048,2009,dcid:Area_LandCover_Forest,48855,SquareKilometer, +dcid:wikidataId/Q22048,2011,dcid:Area_LandCover_Forest,48903,SquareKilometer, +dcid:wikidataId/Q22048,2013,dcid:Area_LandCover_Forest,50545,SquareKilometer, +dcid:wikidataId/Q22048,2015,dcid:Area_LandCover_Forest,50460,SquareKilometer, +dcid:wikidataId/Q22048,2017,dcid:Area_LandCover_Forest,51345,SquareKilometer, +dcid:wikidataId/Q22048,2019,dcid:Area_LandCover_Forest,51619,SquareKilometer, +dcid:wikidataId/Q22048,2021,dcid:Area_LandCover_Forest,52282,SquareKilometer, +dcid:wikidataId/Q22048,2023,dcid:Area_LandCover_Forest,52434,SquareKilometer, +dcid:wikidataId/Q66743,2007,dcid:Area_LandCover_Forest,44,SquareKilometer, +dcid:wikidataId/Q66743,2009,dcid:Area_LandCover_Forest,50,SquareKilometer, +dcid:wikidataId/Q66743,2011,dcid:Area_LandCover_Forest,50,SquareKilometer, +dcid:wikidataId/Q66743,2013,dcid:Area_LandCover_Forest,51,SquareKilometer, +dcid:wikidataId/Q66743,2015,dcid:Area_LandCover_Forest,57,SquareKilometer, +dcid:wikidataId/Q66743,2017,dcid:Area_LandCover_Forest,54,SquareKilometer, +dcid:wikidataId/Q66743,2019,dcid:Area_LandCover_Forest,52,SquareKilometer, +dcid:wikidataId/Q66743,2021,dcid:Area_LandCover_Forest,44,SquareKilometer, +dcid:wikidataId/Q66743,2023,dcid:Area_LandCover_Forest,44,SquareKilometer, +dcid:wikidataId/Q22424,2007,dcid:Area_LandCover_Forest,1664,SquareKilometer, +dcid:wikidataId/Q22424,2009,dcid:Area_LandCover_Forest,1664,SquareKilometer, +dcid:wikidataId/Q22424,2011,dcid:Area_LandCover_Forest,1764,SquareKilometer, +dcid:wikidataId/Q22424,2013,dcid:Area_LandCover_Forest,1791,SquareKilometer, +dcid:wikidataId/Q22424,2015,dcid:Area_LandCover_Forest,1771,SquareKilometer, +dcid:wikidataId/Q22424,2017,dcid:Area_LandCover_Forest,1837,SquareKilometer, +dcid:wikidataId/Q22424,2019,dcid:Area_LandCover_Forest,1849,SquareKilometer, +dcid:wikidataId/Q22424,2021,dcid:Area_LandCover_Forest,1847,SquareKilometer, +dcid:wikidataId/Q22424,2023,dcid:Area_LandCover_Forest,1846,SquareKilometer, +dcid:wikidataId/Q1437,2007,dcid:Area_LandCover_Forest,16036,SquareKilometer, +dcid:wikidataId/Q1437,2009,dcid:Area_LandCover_Forest,16036,SquareKilometer, +dcid:wikidataId/Q1437,2011,dcid:Area_LandCover_Forest,16087,SquareKilometer, +dcid:wikidataId/Q1437,2013,dcid:Area_LandCover_Forest,16086,SquareKilometer, +dcid:wikidataId/Q1437,2015,dcid:Area_LandCover_Forest,16106,SquareKilometer, +dcid:wikidataId/Q1437,2017,dcid:Area_LandCover_Forest,16572,SquareKilometer, +dcid:wikidataId/Q1437,2019,dcid:Area_LandCover_Forest,16630,SquareKilometer, +dcid:wikidataId/Q1437,2021,dcid:Area_LandCover_Forest,16632,SquareKilometer, +dcid:wikidataId/Q1437,2023,dcid:Area_LandCover_Forest,16548,SquareKilometer, +dcid:wikidataId/Q1505,2007,dcid:Area_LandCover_Forest,3357,SquareKilometer, +dcid:wikidataId/Q1505,2009,dcid:Area_LandCover_Forest,3359,SquareKilometer, +dcid:wikidataId/Q1505,2011,dcid:Area_LandCover_Forest,3359,SquareKilometer, +dcid:wikidataId/Q1505,2013,dcid:Area_LandCover_Forest,3358,SquareKilometer, +dcid:wikidataId/Q1505,2015,dcid:Area_LandCover_Forest,3353,SquareKilometer, +dcid:wikidataId/Q1505,2017,dcid:Area_LandCover_Forest,3344,SquareKilometer, +dcid:wikidataId/Q1505,2019,dcid:Area_LandCover_Forest,3342,SquareKilometer, +dcid:wikidataId/Q1505,2021,dcid:Area_LandCover_Forest,3353,SquareKilometer, +dcid:wikidataId/Q1505,2023,dcid:Area_LandCover_Forest,3358,SquareKilometer, +dcid:wikidataId/Q1445,2007,dcid:Area_LandCover_Forest,23338,SquareKilometer, +dcid:wikidataId/Q1445,2009,dcid:Area_LandCover_Forest,23551,SquareKilometer, +dcid:wikidataId/Q1445,2011,dcid:Area_LandCover_Forest,23625,SquareKilometer, +dcid:wikidataId/Q1445,2013,dcid:Area_LandCover_Forest,24245,SquareKilometer, +dcid:wikidataId/Q1445,2015,dcid:Area_LandCover_Forest,26208,SquareKilometer, +dcid:wikidataId/Q1445,2017,dcid:Area_LandCover_Forest,26281,SquareKilometer, +dcid:wikidataId/Q1445,2019,dcid:Area_LandCover_Forest,26364,SquareKilometer, +dcid:wikidataId/Q1445,2021,dcid:Area_LandCover_Forest,26511,SquareKilometer, +dcid:wikidataId/Q1445,2023,dcid:Area_LandCover_Forest,26450,SquareKilometer, +dcid:wikidataId/Q677037,2013,dcid:Area_LandCover_Forest,19957,SquareKilometer, +dcid:wikidataId/Q677037,2015,dcid:Area_LandCover_Forest,19854,SquareKilometer, +dcid:wikidataId/Q677037,2017,dcid:Area_LandCover_Forest,20419,SquareKilometer, +dcid:wikidataId/Q677037,2019,dcid:Area_LandCover_Forest,20582,SquareKilometer, +dcid:wikidataId/Q677037,2021,dcid:Area_LandCover_Forest,21279,SquareKilometer, +dcid:wikidataId/Q677037,2023,dcid:Area_LandCover_Forest,21179,SquareKilometer, +dcid:wikidataId/Q1363,2007,dcid:Area_LandCover_Forest,8073,SquareKilometer, +dcid:wikidataId/Q1363,2009,dcid:Area_LandCover_Forest,7985,SquareKilometer, +dcid:wikidataId/Q1363,2011,dcid:Area_LandCover_Forest,7977,SquareKilometer, +dcid:wikidataId/Q1363,2013,dcid:Area_LandCover_Forest,7949,SquareKilometer, +dcid:wikidataId/Q1363,2015,dcid:Area_LandCover_Forest,7890,SquareKilometer, +dcid:wikidataId/Q1363,2017,dcid:Area_LandCover_Forest,7726,SquareKilometer, +dcid:wikidataId/Q1363,2019,dcid:Area_LandCover_Forest,7726,SquareKilometer, +dcid:wikidataId/Q1363,2021,dcid:Area_LandCover_Forest,7680,SquareKilometer, +dcid:wikidataId/Q1363,2023,dcid:Area_LandCover_Forest,7585,SquareKilometer, +dcid:wikidataId/Q1498,2007,dcid:Area_LandCover_Forest,14341,SquareKilometer, +dcid:wikidataId/Q1498,2009,dcid:Area_LandCover_Forest,14341,SquareKilometer, +dcid:wikidataId/Q1498,2011,dcid:Area_LandCover_Forest,14338,SquareKilometer, +dcid:wikidataId/Q1498,2013,dcid:Area_LandCover_Forest,14360,SquareKilometer, +dcid:wikidataId/Q1498,2015,dcid:Area_LandCover_Forest,14401,SquareKilometer, +dcid:wikidataId/Q1498,2017,dcid:Area_LandCover_Forest,14679,SquareKilometer, +dcid:wikidataId/Q1498,2019,dcid:Area_LandCover_Forest,14806,SquareKilometer, +dcid:wikidataId/Q1498,2021,dcid:Area_LandCover_Forest,14927,SquareKilometer, +dcid:wikidataId/Q1498,2023,dcid:Area_LandCover_Forest,15046,SquareKilometer, +dcid:wikidataId/Q1499,2007,dcid:Area_LandCover_Forest,24495,SquareKilometer, +dcid:wikidataId/Q1499,2009,dcid:Area_LandCover_Forest,24495,SquareKilometer, +dcid:wikidataId/Q1499,2011,dcid:Area_LandCover_Forest,24496,SquareKilometer, +dcid:wikidataId/Q1499,2013,dcid:Area_LandCover_Forest,24508,SquareKilometer, +dcid:wikidataId/Q1499,2015,dcid:Area_LandCover_Forest,24272,SquareKilometer, +dcid:wikidataId/Q1499,2017,dcid:Area_LandCover_Forest,24295,SquareKilometer, +dcid:wikidataId/Q1499,2019,dcid:Area_LandCover_Forest,24303,SquareKilometer, +dcid:wikidataId/Q1499,2021,dcid:Area_LandCover_Forest,24327,SquareKilometer, +dcid:wikidataId/Q1499,2023,dcid:Area_LandCover_Forest,24304,SquareKilometer, +dcid:wikidataId/Q1356,2007,dcid:Area_LandCover_Forest,12994,SquareKilometer, +dcid:wikidataId/Q1356,2009,dcid:Area_LandCover_Forest,12994,SquareKilometer, +dcid:wikidataId/Q1356,2011,dcid:Area_LandCover_Forest,12995,SquareKilometer, +dcid:wikidataId/Q1356,2013,dcid:Area_LandCover_Forest,17102,SquareKilometer, +dcid:wikidataId/Q1356,2015,dcid:Area_LandCover_Forest,16826,SquareKilometer, +dcid:wikidataId/Q1356,2017,dcid:Area_LandCover_Forest,16847,SquareKilometer, +dcid:wikidataId/Q1356,2019,dcid:Area_LandCover_Forest,16902,SquareKilometer, +dcid:wikidataId/Q1356,2021,dcid:Area_LandCover_Forest,16782,SquareKilometer, +dcid:wikidataId/Q1356,2023,dcid:Area_LandCover_Forest,16832,SquareKilometer, +dcid:wikidataId/Q200667,2013,dcid:Area_LandCover_Forest,1478,SquareKilometer, +dcid:wikidataId/Q200667,2021,dcid:Area_LandCover_Forest,2279,SquareKilometer, +dcid:wikidataId/Q200667,2023,dcid:Area_LandCover_Forest,2286,SquareKilometer, +dcid:country/IND,2007,dcid:Area_LandCover_Forest,690899,SquareKilometer, +dcid:country/IND,2009,dcid:Area_LandCover_Forest,692394,SquareKilometer, +dcid:country/IND,2011,dcid:Area_LandCover_Forest,692027,SquareKilometer, +dcid:country/IND,2013,dcid:Area_LandCover_Forest,698712,SquareKilometer, +dcid:country/IND,2015,dcid:Area_LandCover_Forest,701495,SquareKilometer, +dcid:country/IND,2017,dcid:Area_LandCover_Forest,708273,SquareKilometer, +dcid:country/IND,2019,dcid:Area_LandCover_Forest,712249,SquareKilometer, +dcid:country/IND,2021,dcid:Area_LandCover_Forest,715186,SquareKilometer, +dcid:country/IND,2023,dcid:Area_LandCover_Forest,715343,SquareKilometer, diff --git a/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/validation_config.json b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/validation_config.json new file mode 100644 index 0000000000..ae3adf75af --- /dev/null +++ b/statvar_imports/database_on_indian_economy/india_rbi_state_statistics/validation_config.json @@ -0,0 +1,31 @@ +{ + "schema_version": "1.0", + "rules": [ + { + "rule_id": "check_deleted_records_percent", + "description": "Checks that the percentage of deleted records is within the 0.1% threshold. Initial baseline rollout accounts for one-time historical revisions and remappings and official RBI retrospective revisions. Future automated runs strictly enforce <= 0.1% against the promoted baseline.", + "validator": "DELETED_RECORDS_PERCENT", + "params": { + "threshold": 0.1 + } + }, + { + "rule_id": "check_expected_statvar_count", + "description": "Verifies that summary_report.csv is non-empty and contains at least one StatVar.", + "validator": "SQL_VALIDATOR", + "params": { + "query": "SELECT COUNT(DISTINCT StatVar) AS statvar_cnt FROM stats", + "condition": "statvar_cnt >= 1" + } + }, + { + "rule_id": "check_max_date_freshness", + "description": "Verifies that each table meets its expected freshness threshold according to RBI publishing cadence.", + "validator": "SQL_VALIDATOR", + "params": { + "query": "SELECT StatVar, CASE WHEN StatVar IN ('Length_Transportation_Road', 'Length_Transportation_StateHighway') THEN MAX(MaxDate) >= '2020' WHEN StatVar IN ('Consumption_FarmInventory_Fertilizer') THEN MAX(MaxDate) >= '2022' WHEN StatVar IN ('Area_LandCover_Forest', 'IndexScore_SustainableDevelopmentGoal', 'Area_LandCover_Tree', 'Loss_Electricity_TransmissionAndDistribution_AsAFractionOf_Annual_Generation_Electricity', 'GrossValue_Area_LandCover_Cropland', 'NetValue_Area_LandCover_Cropland', 'Gross_Area_LandCover_IrrigatedLand', 'NetValue_Area_LandCover_IrrigatedLand', 'GrossValue_Area_LandCover_Cropland_AsAFractionOf_NetValue_Area_LandCover_Cropland') THEN MAX(MaxDate) >= '2023' ELSE MAX(MaxDate) >= '2024' END AS is_fresh, COUNT(*) AS row_cnt FROM stats WHERE MaxDate IS NOT NULL GROUP BY StatVar", + "condition": "is_fresh = TRUE AND row_cnt > 0" + } + } + ] +}