Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Chuck Connell
How about a handicap fishing pier?
I saw a proposal for a new fishing pier a few weeks ago near the causeway that doesn’t have any public parking and I’ve always thought a handicapped pier on the north side near the school would be a great place to have one. Great parking, level ground access and already has a point to start a pier and it would be out of the predominant west wind most of the time. What are your thoughts? Chuck G.
Does Rain Predict Rain? US Weather Data and the Correlation of Rain Today and Tomorrow
During a dreary stretch of June and July in Boston, it seemed to rain every time my family had planned to do something fun. We started to wonder if we were stuck in a rainy pattern and asked, "Does the fact that it has rained a lot for three days straight, make it likely to rain tomorrow?" I realized that this question is easy to answer using available weather data. This article presents the US weather datasets I used, the Python/pandas program I wrote to analyze the data, and the results.
Does Rain Predict Rain? US Weather Data and the Correlation of Rain Today and Tomorrow
Introducing useful climate datasets and validating a global warming prediction During a dreary stretch of June and July in Boston, it seemed to rain every time my family had planned to do something fun. We started to wonder if we were stuck in a rainy pattern and asked, “Does the fact that it has rained a lot for three days straight, make it likely to rain tomorrow?” I realized that this question is easy to answer using available weather data.
Global.health is Failing
The COVID-19 dataset Global.health (G.h) began almost three years ago with great promise. It is funded by Google.org, Rockefeller Foundation, Boston Children's Hospital and others, with academic support from University of Oxford, Northeastern University, Harvard Medical School and others. Its salient feature is individual person-level data for COVID-19 cases worldwide.
Choropleth Maps with Pandas and Flourish
A tutorial that makes a map of US polio vaccine rates using Python, pandas and Flourish In a previous article, I covered briefly how to make a choropleth map of U.S. county data related to wastewater tracking. The methods used there can be applied to other data, of course, and there are many mapping options that were not shown in those instructions.
Wastewater RNA + Time = Very Strong Outcome Prediction
Combining wastewater RNA level with overall pandemic time yields very high predictions of COVID-19 disease outcomes In a previous article, I showed that the Biobot dataset of SARS-CoV-2 wastewater levels provides strong predictions of COVID-19 disease outcomes in the United States. But there are likely more factors in predicting hospitalization and death than just RNA markers in sewage. Some obvious candidates are vaccination, previous infection, and advances in medical treatment.
Strong Correlation of Wastewater COVID-19 Signal to Hospitalization and Death
The Biobot dataset of USA wastewater strongly predicts COVID-19 disease outcomes with high correlation values. Full Python/pandas code. The USA wastewater dataset from Biobot was joined with COVID-19 disease data. Outcome measures were adjusted in time from the water sample date to match disease progression — 14 days for hospitalization and 28 days for death. The data pairs of water sample to outcome were analyzed for correlation using the standard Spearman rank method.
COVID-19 Outcome Predictions from Wastewater
Forecasting cases, hospitalization and deaths with CDC’s wastewater data. Uses pandas windowing and correlation. More than 50,000 wastewater tests across the United States were matched with COVID-19 outcomes (cases, hospitalization, death) from the same location and timeframe. The resulting dataset was analyzed for correlation between SARS-CoV-2 RNA level and the outcomes, showing a positive relationship with all three measures.
Mapping SARS-CoV-2 Wastewater Coverage
Data engineering to create a Flourish map of water testing by US county. Includes pandas set() operations. Image by Author Wastewater (sewer) data can be used as an early warning for COVID-19 cases. This is a well-developed technology that counts fragments of SARS-CoV-2 RNA in the wastewater, compares the count to previous values in the same region to detect increase/decrease, and can even sequence the RNA to determine virus variants.
SARS-CoV-2 Wastewater Data, Enhanced
Photo by Fusion Medical Animation on Unsplash There has been a lot of press coverage lately about tracking the SARS-CoV-2 virus, which causes the COVID-19 disease, in wastewater (sewer systems). A strong link has been established between spikes in wastewater virus and spikes in disease about a week later. As just one example, this chart shows the virus signal (dark blue) across various water treatment facilities in the United States versus known US COVID-19 cases (light blue).
The Ultimate State-County-FIPS Tool
Photo by KOBU Agency on Unsplash When doing data science with geographic locations it is common to juggle USA FIPS codes, state names, county names, abbreviations, etc. There are various tables of this information scattered around the Internet, but every one I found is lacking in some way. The state abbreviations are missing, or the county names are in inconsistent format, or the table is visual and not easy to download for programmatic use, or some other problem.
JSON Schema: Integrity Checking for NoSQL Data
Photo by Vipul Jha on Unsplash Relational databases have schema. This allows you to always know that the data you have is in the format you expect. For example, both the FIRST_NAME and LAST_NAME fields must have non-null string values, and SALARY must contain an integer greater than 0. Incoming data that does not match the schema is rejected. But many modern data formats, such as JSON, arenot relational and do not enforce schemas.
JSON in Databricks and PySpark
Photo by Fatos Bytyqi on Unsplash In the simple case, JSON is easy to handle within Databricks. You can read a file of JSON objects directly into a DataFrame or table, and Databricks knows how to parse the JSON into individual fields. But, as with most things software-related, there are wrinkles and variations. This article shows how to handle the most common situations and includes detailed coding examples.
Pandas on Spark: Current Issues and Workarounds
Photo by Zan on Unsplash Pandas is a key tool for data analytics and data science and has been around for more than ten years. It is stable and proven. But pandas has a significant limitation that every data engineer bumps into at some point — it runs on just one computer. The data size limit for pandas is approximately 100M rows or 100GB, and these limits can be reached only on a powerful, expensive computer.
Bedford Turkey Trot to be held Thanksgiving morning
The annual Bedford Turkey Trot run and walk will take place on Thanksgiving morning, Nov. 22. All money beyond expenses will go directly to the Bedford Food Pantry. The event features a 5K run and a 2-mile walk. There will be a variety of fun prizes, including for fastest woman and man, most family members, youngest competitor and best turkey costume. Everyone who registers by Oct. 31 gets a high-quality long-sleeve t-shirt.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Chuck
Contact Chuck, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack