Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Julia Silge
Text mining with R: a tidy approach
Much of the data available today is unstructured and text-heavy, making it challenging for analysts to apply their usual data wrangling and visualization tools. With this practical book, you’ll explore text-mining techniques with tidytext, a package that authors Julia Silge and David Robinson developed using the tidy principles behind R packages like ggraph and dplyr. You’ll learn how tidytext and other tidy tools in R can make text analysis easier and more effective.
Delete all your tweets using rtweet
If you are involved in data science social media, I expect that you are quite familiar with the turmoil on Twitter; it doesn’t look like it is getting better. Twitter, and specifically the data science community on Twitter, has brought so much positive to my life and career, and I feel pretty sad about the situation, both the massive layoffs and what looks like a real change for the worse in the platform.
Tidy Modeling with R
Book description Models can be used in almost any domain for purposes including prediction, inference, or simply describing data. In all these cases, the predictive capacity of a model can be used to evaluate it, and we can build better, more useful models by adhering to good statistical practice. The tidymodels framework harmonizes the heterogeneous model interfaces in R and offers a consistent, flexible framework for modeling suitable for beginners as well as the very experienced.
Spatial resampling for #TidyTuesday and the #30DayMapChallenge
This is the latest in my series of screencasts demonstrating how to use the tidymodels packages, from just getting started to tuning more complex models. Today’s screencast walks through how to use spatial resampling for evaluating a model, with this week’s #TidyTuesday dataset on geographic data. 🗾 Here is the code I used in the video, for those who prefer reading instead of or in addition to video.
Caret vs. tidymodels — Create complete reusable machine learning workflows
Photo by Jonathan Tomas on Unsplash If you use machine learning models in R, you probably use either caret or tidymodels. Interestingly, both packages were developed by the same author among many others: Max Kuhn. But how do they compare to each other in terms of feasibility and performance? You may wonder which package you should learn for predictive modelling in R.
Supervised Machine Learning for Text Analysis in R
Supervised Machine Learning for Text Analysis in R ISBN | Quantity: Shopping Cart Summary Items: Routledge & CRC Press eBooks are available through VitalSource. The free VitalSource Bookshelf® application allows you to access to your eBooks whenever and wherever you choose. Mobile/eReaders – Download the Bookshelf mobile app at VitalSource.com or from the iTunes or Android store to access your eBooks from your mobile device or eReader.
Dimensionality reduction for #TidyTuesday Billboard Top 100 songs
This is the latest in my series of screencasts demonstrating how to use the tidymodels packages, from just getting started to tuning more complex models. Today’s screencast focuses only on data preprocessing, or feature engineering; let’s walk through how to use dimensionality reduction for song features sourced from Spotify (mostly audio), with this week’s #TidyTuesday dataset on Billboard Top 100 songs.
Supervised Machine Learning for Text Analysis in R is now complete
Last summer, Emil Hvitfeldt and I announced that we had started work on a new book project, to be published in the Chapman & Hall/CRC Data Science Series, and we are now happy to say that Supervised Machine Learning for Text Analysis for R (or SMLTAR, as we call it for short) is complete, in production, and available for preorder! You should be able to preorder it anywhere you normally buy books, such as Bookshop, Amazon, or directly from our publisher.
Tune xgboost models with early stopping to predict shelter animal status
This is the latest in my series of screencasts demonstrating how to use the tidymodels packages, from just getting started to tuning more complex models. I participated in this week’s episode of the SLICED playoffs, a competitive data science streaming show, where we competed to predict the status of shelter animals.
Supervised Machine Learning for Text Analysis in R
Text data is important for many domains, from healthcare to marketing to the digital humanities, but specialized approaches are necessary to create features for machine learning from language. Supervised Machine Learning for Text Analysis in R explains how to preprocess text data for modeling, train models, and evaluate model performance using tools from the tidyverse and tidymodels ecosystem.
Use racing methods to tune xgboost models and predict home runs
This is the latest in my series of screencasts demonstrating how to use the tidymodels packages, from just getting started to tuning more complex models. This week’s episode of SLICED, a competitive data science streaming show, had contestants compete to predict home runs in recent baseball games. Honestly I don’t know much about baseball ⚾ but the finetune package had a recent release and this challenge offers a good opportunity to show how to use racing methods for tuning.
Create a custom metric with tidymodels and NYC Airbnb prices
This is the latest in my series of screencasts demonstrating how to use the tidymodels packages, from just getting started to tuning more complex models. This week’s episode of SLICED, a competitive data science prediction challenge, introduced a challenge for predicting the prices of Airbnb listings in NYC. In today’s screencast, I walk through how to build such a model combining tabular data with unstructured text data from the listing names, and how to create a custom metric in tidymodels.
Class imbalance and classification metrics with aircraft wildlife strikes
This is the latest in my series of screencasts demonstrating how to use the tidymodels packages, from just starting out to tuning more complex models with many hyperparameters. I recently participated in SLICED, a competitive data science prediction challenge. I did not necessarily cover myself in glory but in today’s screencast, I walk through the data set on aircraft wildlife strikes we used and how different choices around handling class imbalance affect different classification metrics.
Model Monitoring with R Markdown, pins, and RStudio Connect
Julia Silge 2021-04-08 Categories: RStudio Connect R Markdown Data Science Leadership Tags: machine learning modeling rmarkdown flexdashboard pins ModelOps or MLOps (for “model/machine learning operations”) focuses on the real-world processes involved in building, deploying, and maintaining a model within an organization’s data infrastructure.
Learn tidytext with my new learnr course
Today I am happy to announce that a new free, online, open source, interactive tutorial, Text Mining with Tidy Data Principles, has been published! 🎉 I previously developed an interactive course on text mining for an online learning company, but that course is no longer available.
1 The tidy text format
Using tidy data principles is a powerful way to make handling data easier and more effective, and this is no less true when it comes to dealing with text. As described by Hadley Wickham (Wickham 2014), tidy data has a specific structure: Each variable is a column Each observation is a row Each type of observational unit is a table We thus define the tidy text format as being a table with one-token-per-row.
rstudio::global(2021) Diversity Scholarships
Julia Silge and Jen Hecht 2020-11-30 Categories: Company News & Events Events Tags: diversity inclusion rstudio::global(2021) will be a very different conference from previous years. We will miss being together physically, but we are enthusiastic about planning this free, virtual event designed to be inclusive of R users in every time zone. Even though the conference itself is free, we are continuing our tradition of diversity scholarships, but with a different focus.
Supervised Machine Learning for Text Analysis in R
Today, Emil Hvitfeldt and I led a useR! 2020 online tutorial on predictive modeling with text using tidy data principles. This tutorial was hosted by R-Ladies en Argentina; huge thanks to the organizers for their leadership and effort in making this tutorial possible.
Learn tidymodels with my supervised machine learning course
Today I am happy to announce that a new tidymodels-centric version of my free, online, interactive course, Supervised Machine Learning: Case Studies in R, has been published! 🎉 This is at least the third version of this course I’ve built at this point 😁 but I believe it to be the best, in terms of how it communicates machine learning concepts and how useful to your real-world problems the demonstrated code will be.
Research update: Coding on the Weekends
Welcome to this month’s installment of Stack Overflow research updates! I code for work and for fun, on weekdays here at Stack Overflow as a data scientist and on the weekends as an open source maintainer. This is not uncommon. About say they code as a hobby, and on my team, it turns out that every single one of us does a bit of coding for fun, whether that’s live coding on Twitch, connecting C# to R Markdown because you can, or learning game development from scratch.
Coding Salaries in 2019: Updating the Stack Overflow Salary Calculator
Today we are launching the 2019 update to the Stack Overflow Salary Calculator, a tool that allows developers and employers to understand representative salaries for the software industry based on experience level, location, education, and specific technologies. This is the third year we have released our salary calculator, and we continue to use the same modeling approach from previous years.
Stack Speaks: Talking text mining and developer data on the Cause A Scene podcast
Kim Crayton from #causeascene podcast. Stack Speaks is a chance for us to highlight interesting appearances, talks, and presentations from Stack Overflow employees. Recently, I sat down with Kim Crayton as a guest on her #causeascene podcast. Kim is a coach, speaker, and leader who helps tech organizations put their values into operation and learn how to identify organizational barriers to inclusion.
Public Data Release of Stack Overflow’s 2019 Developer Survey
Last month, we published the results from Stack Overflow’s 2019 Developer Survey, and today you can access the data from this year’s survey for yourself. Almost 90,000 developers from around the world shared their opinions about the jobs they have, the technologies they use everyday, and where they see the industry going. This year, we are partnering with Glitch to publish and highlight this data. This means you can access the data here on our site, and you can explore the dataset on Glitch.
The 2019 Stack Overflow Developer Survey Results Are In
We’ve been working hard for months, and today the Developer Survey results for 2019 are live. This year marks the ninth year we’ve published our annual Developer Survey, and nearly 90,000 developers took the 20-minute survey earlier this year. See the results for yourself! What are some of this year’s key results?
2 Sentiment analysis with tidy data
In the previous chapter, we explored in depth what we mean by the tidy text format and showed how this format can be used to approach questions about word frequency. This allowed us to analyze which words are used most frequently in documents and to compare documents, but now let’s investigate a different topic. Let’s address the topic of opinion mining or sentiment analysis.
Welcome Wagon: Community and Comments on Stack Overflow
This past summer, we wrote our first blog post about comments on Stack Overflow, focusing on our initial work rating comments internally at Stack Overflow and what we learned. Since then, we’ve fielded this comment rating task more broadly in our community. This blog post shares some of what we are learning.
Text Mining with R
So far we’ve considered words as individual units, and considered their relationships to sentiments or to documents. However, many interesting text analyses are based on the relationships between words, whether examining which words tend to follow others immediately, or that tend to co-occur within the same documents. In this chapter, we’ll explore some of the methods tidytext offers for calculating and visualizing relationships between words in your text dataset.
Developer Salaries in 2018: Updating the Stack Overflow Salary Calculator
Today we launched the 2018 update to the Stack Overflow Salary Calculator, a tool that allows developers and employers to find typical salaries for the software industry based on experience level, location, education, and specific technologies. The methodology we used is similar to last year, but this year we’ve added support for eight new countries and refined which technologies contribute to our salary predictions.
The pick of R packages
Our August 2018 cover story celebrates the 25th anniversary of the creation of R, the statistical programming language that became a subcultural phenomenon. While researching R’s history, community and culture, our reporter Nick Thieme interviewed a number of active R users and developers – and he invited them to tell our readers about some of the things they love most about R: specifically, the packages.
Welcome Wagon: Classifying Comments on Stack Overflow
Last month, Joe wrote about the Welcome Wagon work that we are doing to make Stack Overflow more welcoming and inclusive. Our current work involves projects across domains from asking questions to framing community standards and more; one project we have been working on is understanding how comments are used and misused on Stack Overflow. We are a data engineer (Jason) and data scientist (Julia).
Public Data Release of Stack Overflow’s 2018 Developer Survey
Starting today, you can access the public data release for Stack Overflow’s 2018 Developer Survey. Over 100,000 developers from around the world shared their opinions about everything from their favorite technologies to job preferences, and this data is now available for you to analyze yourself. This year, we are partnering with Kaggle to publish and highlight this dataset.
Information Architecture for All: Navigating Stack Overflow, Enterprise, and Stack Exchange Sites - Stack Overflow Blog
This post is part of a series on how we’re making Channels, the thinking behind the product, and insight into the process. Read “How We’re Designing Channels” and “Why Channels” for more background info. In his post, How We’re Designing Channels, Kurtis wrote that this project required a change to Stack Overflow’s information architecture. We created several prototype navigations, narrowed to two, and tested with a group of users.
New sports from random emoji
I love emoji ❤️ and I love xkcd, so this recent comic from Randall Munroe was quite a delight for me. I sat there, enjoying the thought of these new sports like horse hole and multiplayer avocado and I thought, “I can make more of these in just the barest handful of lines of code”. This is largely thanks to the emo package by Hadley Wickham, which if you haven’t installed and started using yet, WHY NOT???
Dramatic Shifts in Technologies on Stack Overflow | Stack Overflow Blog
Life as a developer (or data scientist, in my case) involves being comfortable with changing technologies. I don’t use the same programming languages that I did at the beginning of my career, and I fully expect to be using different technologies several years from now.
3 Ways You Can Be an Ally to Women in Tech - Stack Overflow Blog
According to the National Center for Women & Information Technology, only 26% of professional computing roles in the workforce in the US are held by women. Within prominent companies in the industry, the numbers tend to be even lower. It’s clear that there’s work to be done. Many organizations are putting in the work to increase diversity, but in the face of omnipresent cultural touchpoints like this tirade from a former Google employee, the battle feels more uphill than ever.
How Much Do Developers Earn? Find Out with the Stack Overflow Salary Calculator - Stack Overflow Blog
Today, we launched the Stack Overflow Salary Calculator, a tool that allows developers and employers to find typical salaries based on experience level, location, specific technologies, and education. We did this because we believe developers should be empowered with more information around job searches, careers, and salary.
Gender Roles with Text Mining and N-grams
Today is the one year anniversary of the janeaustenr package’s appearance on CRAN, its cranniversary, if you will. I think it’s time for more Jane Austen here on my blog. I saw this paper by Matthew Jockers and Gabi Kirilloff a number of months ago and the ideas in it have been knocking around in my head ever since. The authors of that paper used text mining to examine a corpus of 19th century novels and explore how gendered pronouns (he/she/him/her) are associated with different verbs.
And the Most Realistic Developer in Fiction is... - Stack Overflow Blog
Computer programmers, IT specialists, and hackers are often depicted in popular works of fiction, from books, movies, to TV shows. On the 2017 Developer Survey, we asked respondents about depictions of what it’s really like to be a programmer. This was one of my favorite questions on the survey this year, and analyzing these results has been quite the glimpse into the opinions of our community.
What Programming Languages Are Used Most on Weekends?
For me, the weekends are mostly about spending time with my family, reading for leisure, and working on the open-source projects I am involved in. These weekend projects overlap with the work that I do in my day job here at Stack Overflow, but are not exactly the same. Many developers tinker with side projects for learning or career development (or just for fun!) and at Stack Overflow, we support all types of technologies, from professional to hobbyist.
Tidy Text Mining with R
I am so pleased to announce that tidytext 0.1.2 is now available on CRAN. This release of tidytext, a package for text mining using tidy data principles by Dave Robinson and me, includes some bug fixes and performance improvements, as well as some new functionality. There is now a handy function for accessing the various lexicons in the dataset without the columns that are not used in that particular dataset; this makes these datasets even easier to use with pipes and joins from dplyr.
The Life-Changing Magic of Tidying Text
When I went to the rOpenSci unconference about a month ago, I started work with Dave Robinson on a package for text mining using tidy data principles. What is this tidy data you keep hearing so much about? As described by Hadley Wickham, tidy data has a specific structure: This means we end up with a data set that is in a long, skinny format instead of a wide format. Tidy data sets are easier to work with, and this is no less true when one starts to work with text.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Julia
Contact Julia, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack