Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Erdogan Taskesen
A Practical Starters’ Guide to Causal Structure Learning with Bayesian Methods in Python
Privacy Preference Center When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies.
Prescriptive Modeling Unpacked: A Complete Guide to Intervention With Bayesian Modeling.
In this article, I will demonstrate how to move from simply forecasting outcomes to actively intervening in systems to steer toward desired goals. With hands-on examples in predictive maintenance, I will show how data-driven decisions can optimize operations and reduce downtime. with descriptive analysis to investigate “what has happened”. In predictive analysis, we aim for insights and determine “what will happen”.
How to Generate Synthetic Data: A Comprehensive Guide Using Bayesian Sampling and Univariate Distributions
that drives organizations nowadays. But what happens when observations are scarce, costly, or hard to collect? That’s where synthetic data comes into play because we can generate artificial data that mimics the statistical properties of real-world observations. In this blog, I will provide a background in synthetic data, together with practical hands-on examples. I will discuss two powerful techniques on how to generate synthetic data: Bayesian Sampling and Univariate Distribution Sampling.
K-Nearest-Rank Scores: A Tool for Local Structure Analysis in Embeddings
When using a low-dimensional projection, we should always be aware that it is merely a projection of a higher and more complex dimensional space. To truly trust these projections, we need to assess how well they preserve the original sample distribution. Dimensionality reduction techniques such as PCA, t-SNE, and UMAP are widely used for both feature reduction and visualizing data in lower-dimensional spaces.
Building a Custom AI Jira Agent
The inspiration for this project came from hosting a Jira ticket creation tool on a web application I had developed for internal users. I also added automated Jira ticket creation upon system errors. Users and system errors often create similar tickets, so I wanted to see if the reasoning capabilities of LLMs could be used to automatically triage tickets by linking related issues, creating user stories, acceptance criteria, and priority.
An Extensive Starters Guide For Causal Discovery using Bayesian Modeling
Bayesian approaches are becoming increasingly popular but can be overwhelming at the start. This extensive guide will walk you through applications, libraries, and dependencies of causal discovery approaches. The endless possibilities of Bayesian techniques are also their weakness; the applications are enormous, and it can be troublesome to understand how techniques are related to different solutions and thus applications.
An Extensive Starters Guide For Causal Discovery using Bayesian Modeling
An Extensive Starters Guide For Causal Discovery using Bayesian Modeling The endless possibilities of Bayesian techniques are also their weakness; the applications are enormous, and it can be troublesome to understand how techniques are related to different solutions and…
An Extensive Starters Guide For Causal Discovery using Bayesian Modeling
An Extensive Starter Guide For Causal Discovery Using Bayesian Modeling . This extensive guide will walk you through applications, libraries, and dependencies of causal discovery approaches. The endless possibilities of Bayesian techniques are also their weakness; the applications are enormous, and it can be troublesome to understand how techniques are related to different solutions and thus applications.
All You Need Is Statistics to Analyze Tabular Datasets
To analyze tabular datasets there is no need for deep learning nor large language models. I will demonstrate how (simple) statistics, and techniques such as PCA can show new insights and explainable results. Tabular datasets are one of the most common forms of data and consist of a mix of variables such as binary, categorical, textual, and continuous values. A well-known tabular dataset is, for example, the Titanic dataset.
What You Need To Know To Build Large Streamlit Applications With Stripe Subscriptions And Firestore Integration.
The ability to turn ideas into software products is a great skill to learn. In this blog, I will describe what it takes, and how to put the parts together to create a software product without starting costs but with a subscription model and Firestore integration. Whether you are a data scientist, data engineer, or in another field of software development, turning your thoughts into real working software products using only a laptop may be the greatest skill to have.
To Data or Not to Data.
The question is not anymore whether we can solve the problem with AI but to what extent it returns sustainable and reliable results. Good craftsmanship, governance, ethics, and education on AI are what we need now. Since I was a kid, I have always been intrigued and interested in new technology and today we live in a world where smart and connected devices have become a part of our daily lives.
Chat with Your Dataset using Bayesian Inferences.
The ability to ask questions to your data set has always been an intriguing prospect. You will be surprised how easy it is to learn a local Bayesian model that can be used to interrogate your data set. With the rise of chatGPT-like models, it has become accessible for a broader audience to analyze your own data set and, so to speak, “ask questions”. Although this is great, such an approach has also disadvantages when using it as an analytical step in automated pipelines.
Chat with Your Dataset using Bayesian Inferences.
With the rise of chatGPT-like models, it has become accessible for a broader audience to analyze your own data set and, so to speak, " ask questions". Although this is great, such an approach has also disadvantages when using it as an analytical step in automated pipelines. This is especially the case when the outcome of models can have a significant impact. To maintain control and ensure results are accurate we can also use Bayesian inferences to talk to our data set.
Detection of Multicollinearity in Data sets using Statistical Testing.
Detecting multicollinearity in data sets is an important step but also challenging. I will demonstrate how to detect variables with similar behavior in mixed data sets and how to deeper examine the relationships with interactive charts. Understanding the strength of relationships between variables in a data set is important because variables with statistically similar behavior can affect the reliability of models.
A Step-by-Step Guide To Accurately Detect Peaks and Valleys.
Peak Detection is a challenging step in many applications. Read and learn how to accurately detect peaks and valleys in 1D vectors and 2D arrays (images). Our human brain is excellent in peak detection in relation to its context. What seems an easy task by eye can be a challenging task to automate by machines. In general, peaks and valleys indicate (significant) events such as sudden increases or decreases in price/volume, or sharp rises in demand.
The Next Step is Responsible AI. How Do We Get There? Original
Machine learning solutions take an important place in our lives. It is not only about performance anymore but also about responsibility. In the last decades, many AI projects focused on model efficiency and performance. Results are documented in scientific articles, and the best-performing models are deployed in organizations. Now it is the time to put another important part into our AI systems; responsibility.
Maximize Your Insights by Choosing the Best Chart: Network, Heatmap, or Sankey?
Beautiful visualizations are great but to maximize the interpretability, you need to choose a chart carefully. Visualization is an important part of data analysis as it can transform data into insights and help you with storytelling. In this blog post, I will focus on Network charts, Heatmaps, and Sankey charts. These charts have the same input, but we should keep in mind that they are designed with a specific goal, and the interpretability can therefore differ.
Effectively Optimize Your Regression Model with Bayesian Hyperparameter Tuning
Learn to effectively optimize hyperparameters, and prevent creating overtrained models for XGBoost, CatBoost, and LightBoost Gradient boosting techniques such as XGBoost, CatBoost, and LightBoost has gained much popularity in recent years for both classification and regression tasks. An important part of the process is the tuning of hyperparameters to gain the best model performance.
The Path to Success in Data Science Is About Your Ability to Learn. But What to Learn?
The Path to Success in Data Science Is About Your Ability to Learn. But What to Learn? The chances of successfully delivering data science projects are greatest when you keep learning, but it’s not always clear what to focus on Photo by Tamara Menzi on Unsplash Many great developments in data science have been made in the last decade but despite…
The Path to Success in Data Science Is About Your Ability to Learn. But What to Learn?
The chances of successfully delivering data science projects are greatest when you keep learning, but it's not always clear what to focus on Many great developments in data science have been made in the last decade but despite these achievements, many projects never see the light of day.
Create and Explore the Landscape of Roles and Salaries in Data Science
Create and Explore the Landscape of Roles and Salaries in Data Science A tutorial on how to create a landscape using categorical data, and perform unsupervised analysis for deeper insights Image by Author. The data science field is under constant development for which new roles and functions are created. The traditional data science role is evolving into…
Create and Explore the Landscape of Roles and Salaries in Data Science
A tutorial on how to create a landscape using categorical data, and perform unsupervised analysis for deeper insights The data science field is under constant development for which new roles and functions are created. The traditional data science role is evolving into tens of new roles, from Data Engineer, ML Engineer, Product Data Analyst, Research Scientist, Cloud Data Engineer, and many more.
The Power of Bayesian Causal Inference: A Comparative Analysis of Libraries to Reveal Hidden Causality in Your Dataset.
Reveal the hidden causal variables in your data set by using the best-suited Bayesian causal inference library: a comparison with hands-on examples of five popular libraries. Understanding the causal effect of variables in systems or processes is very valuable. There are a number of Python libraries that can assist in determining causal relationships. I will compare five popular causal inference libraries in their functionality, ease of use, and flexibility. Each is accompanied by hands-on examples.
From Clusters To Insights; The Next Step
Learn how to quantitatively detect which features drive the formation of the clusters Cluster analysis is a great technique for identifying groups with similar patterns. However, once clusters are formed, it can remain challenging to determine the driving features behind the clusters. But this step is crucial to reveal valuable insights that may have been missed before and can be used for decision-making and a deeper understanding of your data set.
From Clusters To Insights; The Next Step
For this use case, we will load the online shoppers' intentions data set and go through the steps of preprocessing, clustering, evaluation and then determining the significantly associated features for the cluster labels. This data set contains in total of 12330 samples with 18 features. This mixed dataset requires a few more pre-processing steps to make sure that all variables have similar types or units of measurement.
From Data to Clusters: When is Your Clustering Good Enough?
Hidden gems can be found using clustering approaches but you need the right clustering method and evaluation approach to make sensible clusters. Learn how to find them in four steps. With unsupervised cluster analysis, we can group observations with similar patterns, and reveal (hidden) trends in the data. The use of cluster evaluation methods helps to determine the clustering tendency, quality, and optimal number of clusters.
From Data to Clusters: When is Your Clustering Good Enough?
Idaho Murder Suspect Bryan Kohberger Claims Surviving Roommate Has Evidence That May Help Clear His Name
From Data to Clusters; When is Your Clustering Good Enough?
Hidden gems can be found using clustering approaches but you need the right clustering method and evaluation approach to make sensible clusters. Learn how to find them in four steps. With unsupervised cluster analysis, we can group observations with similar patterns, and reveal (hidden) trends in the data. The use of cluster evaluation methods helps to determine the clustering tendency, quality, and optimal number of clusters.
Step-by-Step Guide to Generate Synthetic Data by Sampling From Univariate Distributions.
Learn how to create synthetic data in case your project runs low on data or use it for simulations Data is the fuel in Data Science projects. But what if the observations are scarce, expensive, or difficult to measure? Synthetic data can be the solution. Synthetic data is artificially generated data that mimics the statistical properties of real-world events. I will demonstrate how to create continuous synthetic data by sampling from univariate distributions.
Step-by-Step Guide to Generate Synthetic Data by Sampling From Univariate Distributions.
Learn how to create synthetic data in case your project runs low on data or use it for simulations Data is the fuel in Data Science projects. But what if the observations are scarce, expensive, or difficult to measure? Synthetic data can be the solution. Synthetic data is artificially generated data that mimics the statistical properties of real-world events. I will demonstrate how to create continuous synthetic data by sampling from univariate distributions.
Outlier Detection Using Principal Component Analysis and Hotelling’s T2 and SPE/DmodX Methods
Thanks to PCA’s sensitivity, it can be used to detect outliers in multivariate datasets. Principal Component Analysis (PCA) is a widely used technique for dimensionality reduction while preserving relevant information. Due to its sensitivity, it can also be used to detect outliers in multivariate datasets. Outlier detection can provide early warning signals for abnormal conditions, allowing experts to identify and address issues before they escalate.
Outlier Detection Using Principal Component Analysis and Hotelling's T2 and SPE/DmodX Methods
Principal Component Analysis (PCA) is a widely used technique for dimensionality reduction while preserving relevant information. Due to its sensitivity, it can also be used to detect outliers in multivariate datasets. Outlier detection can provide early warning signals for abnormal conditions, allowing experts to identify and address issues before they escalate. However, detecting outliers in multivariate datasets can be challenging due to the high dimensionality, and the lack of labels.
Outlier Detection Using Distribution Fitting in Univariate Datasets
Learn how to detect outliers using Probability Density Functions for fast and lightweight models and explainable results. Anomaly or novelty detection is applicable in a wide range of situations where a clear, early warning of an abnormal condition is required, such as for sensor data, security operations, and fraud detection among others.
How to Find the Best Theoretical Distribution for Your Data
Knowing the underlying data distribution is an essential step for data modeling and has many applications, such as anomaly detection, synthetic data creation, and data compression. Knowing the underlying (probability) distribution of your data has many modeling advantages. The easiest manner to determine the underlying distribution is by visually inspecting the random variable(s) using a histogram.
The Starters Guide to Release your Python Package in PyPi.
A step-by-step guide to effectively release your Python package in the Python Package Index (PyPI) to pip install it Releasing a package in PyPi and installing it with pip has great advantages as it will automatically resolve dependencies, and it becomes accessible on any machine. However, publishing an open-source Python Package requires some preparation steps, such as the unambiguous structuring of your directory, versioning, archive building, and publishing the package.
Get the most out of your Scatterplot by making it interactive using D3js and Python.
Scatterplots are extremely useful for visualizing relationships between two sets of numerical variables. It is even more insightful when it is interactive with zooming and brushing capabilities. The scatter plot is perhaps the most well-known chart to visualize numerical variables. Such basic charts are very useful from time to time, especially when they are interactive with brushing and zooming capabilities. The is part of the and can be created using Python.
Hands-on Guide to Create beautiful Sankey Charts in d3js with Python.
To summarize, I demonstrated how the create your own Sankey chart using Python and d3js functionality. The chart is developed in such a manner that it can handle various data sets and allows interactively changing colors, ordering, and positioning of the states. The Sankey chart is one of the blocks in D3Blocks where d3-javascript forms the basis. The use of d3-javascript has many advantages, such as speed, and flexibility, and you don’t need any other technology than a browser to plot the graphs.
How to Create Storytelling Moving Bubbles Charts in d3js with Python.
The MovingBubble chart is one of those mindblowing charts to look at. It is a great way to conceptually better understand how individual items are distributed across states and move across time. Learn how to create them with Python and your own data set. The moving bubble chart is one of those charts that is mind-blowing to look at. It is a great way for storytelling and to conceptually better understand how individual items are distributed across states and move across time.
How to Create Storytelling Moving Bubbles Charts in d3js with Python.
The moving bubble chart is one of those charts that is mind-blowing to look at. It is a great way for storytelling and to conceptually better understand how individual items are distributed across states and move across time. Despite their attractive appearance, such charts are not frequently seen because of the huge effort that needs to be taken to create a single chart. Here comes the library into play because the movingbubbles chart is part of .
The Python Library to Create Interactive and Standalone D3js Charts.
Create interactive, stand-alone, and visually attractive charts that are built on the graphics of d3 javascript (d3js) but configurable with Python. Python has become one of the most popular programming languages to analyze and visualize your data. Visualizing can be the key to success in projects because it can reveal hidden insights in the data, and improve understanding. The best way to understand and explain the data is by making it interactive.
A Guide to Find the Best Boosting Model using Bayesian Hyperparameter Tuning but without Overfitting
With boosted decision tree algorithms, such as XGBoost, CatBoost, and LightBoost you may outperform other models but overfitting is a real danger. Learn how to split the data, optimize hyperparameters, and find the best-performing model without overtraining it using the HGBoost library. Gradient boosting techniques gained much popularity in recent years for classification and regression tasks. An important part is the tuning of hyperparameters to gain the best performance in predictions.
The similarity between t-SNE, UMAP, PCA, and other mappings.
Low dimensional projections are useful to better understand the relationships between samples but how similar is one map to another? The use of dimensionality reduction methods such as PCA, t-SNE, and UMAP are great techniques for feature reduction and visualization of samples in low dimensions. If we can see how samples are distributed by eye, it can provide new key insights. But how representative is a reduced dimensional space compared to an (original) high(er) dimensionality?
A practical guide for getting the most out of Principal Component Analysis
A practical guide for getting the most out of Principal Component Analysis (image by the author) Principal Component Analysis is the most well-known technique for (big) data analysis. However, interpretation of the variance in the low-dimensional space can remain challenging. Understanding the loadings and interpreting the biplot is a must-know part for anyone who uses PCA.
Creating beautiful stand-alone interactive D3 charts with Python
Screenshot from: https://d3js.org/ Visualizing your data can be the key to success in projects because it can reveal hidden insights inthe data, and improve understanding. The best way to convince people is by letting them see and interact with their data. Despite many visualization packages being available in Python, it is not always straightforward to make beautiful stand-alone and interactive charts that can also work outside your own machine.
Detection of duplicate images using image hash functions.
Searching for (near-)identical photos across your entire system to remove duplicates can be a tedious effort; you need to click across thousands of photos, and then decide for each photo whether it is a “similar” one. The most straightforward approach to detect duplicates would be on file size or filename.
A step-by-step guide for clustering images
For the detection and exploration of natural groups or clusters of images by carefully pre-processing images, utilizing well-known feature extraction approaches, and evaluation of the goodness of the clustering. A theoretical background followed by a hands-on tutorial. Photo by Universal Eye on Unsplash Many computer vision tasks rely on (deep) neural networks, and aim to predict “what’s on the image”. However, not all tasks require supervised approaches or neural networks.
A step-by-step guide in designing knowledge-driven models using Bayesian theorem.
Photo by: Kieran Wood, Unsplash Data is the fuel for models but you may have witnessed situations where there is no data but solely a domain expert’s that can very well describe or even predict “the situation” given the circumstances. I will summarize the concepts of knowledge-driven models in terms of Bayesian probabilistic, followed by a hands-on tutorial to demonstrate the steps of converting an expert’s knowledge into a Bayesian model with the goal to make inferences.
A Step-by-Step Guide in detecting causal relationships using Bayesian Structure Learning in Python.
Photo by GR Stocks on Unsplash Determining causality across variables can be a challenging step but is important for strategic actions. I will summarize the concepts of causal models in terms of Bayesian probabilistic, followed by a hands-on tutorial to detect causal relationships using Bayesian structure learning. I will use the sprinkler dataset to conceptually explain how structures are learned with the use of the Python library bnlearn.
Explore and understand your data with a network of significant associations.
Photo by Kurt Cotoaga on Unsplash Understanding your data is an important step in data science. However, if you don’t make any assumptions on the data, it can become a time consuming and challenging step. It is easy to neglect this step and jump too soon in the water. However, it is wise to learn to swim before jumping into the water! This can make the difference between a unsuccessful project or finishing successfully! One manner to explore your data is with the use of the method HNet [1, 2, 3].
Genome-wide association analysis of insomnia complaints identifies risk genes and genetic overlap with psychiatric and metabolic traits
This work was funded by the Netherlands Organization for Scientific Research (NWO Brain & Cognition 433-09-228, NWO VICI 453-14-005 and 453-07-001, 645-000-003) and by the European Research Council (ERC-ADG-2014-671084 INSOMNIA).
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Erdogan
Contact Erdogan, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack