Skip to content

US Census ACS5yr S2409: Clean non-numeric characters in Quantity values - #2265

Open
kartik-s21 wants to merge 3 commits into
datacommonsorg:masterfrom
kartik-s21:s2409-refresh
Open

kartik-s21 wants to merge 3 commits into
datacommonsorg:masterfrom
kartik-s21:s2409-refresh

Conversation

@kartik-s21

@kartik-s21 kartik-s21 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Cleans non-numeric characters (such as commas ,, dashes -, and plus signs + in top/bottom-coded values like 2,500- and 250,000+) from Quantity values in scripts/us_census/acs5yr/subject_tables/s2409/data_loader.py and casts to float during processing for the 2010–2024 S2409 refresh.

Testing

  • Verified unit tests pass: python3 -m unittest scripts/us_census/acs5yr/subject_tables/s2409/subject_table_s2409_test.py

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the data loader to clean quantity values by removing commas, dashes, and plus signs before converting them to floats. Feedback suggests using pd.to_numeric with errors='coerce' and vectorized string operations to prevent potential ValueError crashes from unexpected non-numeric characters.

Comment thread scripts/us_census/acs5yr/subject_tables/s2409/data_loader.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant