Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Necati Demir
Apache Spark Optimization Techniques
Large-scale data analysis has become a transformative tool for many industries, with applications that include fraud detection for the banking industry, clinical research for healthcare, and predictive maintenance and quality control for manufacturing. However, processing such vast amounts of data can be a challenge, even with the power of modern computing hardware.
AI-powered Automatic Speech Recognition: What’s Next? | Toptal®
Term Definition Why It Matters Noise Reduction Techniques used to filter out background sounds using signal processing or learned noise suppression. Improves ASR performance in real-world environments by enhancing audio clarity before transcription begins. Acoustic Model In traditional systems, this component maps audio to phonetic units (e.g., phonemes). In end-to-end models like Whisper, this process is learned implicitly within the neural network. Essential for accurate sound-to-text conversion.
The Silent Killer in Machine Learning: Data Leakage
· What is Data Leakage? · Types of Data Leakage · Why is Data Leakage Dangerous? · How to Prevent Data Leakage · Real-World Examples ∘ Example #1 ∘ Example #2 · Conclusion You have been assigned a new machine learning project, you are excited and you started working on model architectures, hyperparameter tuning, and feature engineering. However, there’s a silent killer that can mess up all of these efforts: data leakage.
N’s Weekly #71: A newsletter about DevOps & Machine Learning & Management
[A] Gitlab CI/CD steps — Gitlab announced CI/CD steps, a programming language for DevOps. As the name clearly Its goal is to help developers to create workflow automation. [T] lakeFS — lakeFS is a tool that transforms your object storage into a Git-like repository. In short, it is Git for data. [A] Docker Best Practices: Understanding the Differences Between ADD and COPY Instructions in Dockerfiles— A tutorial that explains when to use ADD and when to use COPY.
A newsletter about DevOps & Machine Learning & Management
[T] nomad - Nomad is not a new tool but I have never used it. It is workload orchestrator, alternative to Kubernetes, developed by hashicorp. [A] How to load-balance across multiple availability zones for improved redundancy - AWS made our life easy to handle those situations where you need load-balancer across multiple AZs. This tutorial shows how to do it.
A newsletter about DevOps & Machine Learning & Management
[A] Understanding the Docker USER Instruction - You are certainly dealing with Dockerfile if you are a DevOps Engineer. And, there is a high probability that you used USER instruction at some point by copy/pasting it from somewhere ;) But, do we really know the details? This article dives into the details. [A] How To Manage Terraform Versions - The title of the article explain itself well.
A newsletter about DevOps & Machine Learning & Management
[A] 73 Most Useful DevOps Tools to Use in 2024 - A list of DevOps tools you may be interested in learning [A] Kubernetes with GPU for MLOps Workloads - GPU became one of the most important parts of production deployments. Let's also not forget that Kubernetes is still popular. So, this article will help you how to run Kubernetes (to be specific, it is Minikub) with GPU. [A] 26 Top Kubernetes Tools for 2024 - The title explains itself.
A newsletter about DevOps & Machine Learning & Management
[T] StarRocks - StarRocks is an Open-Source, High-Performance Analytical Database. I have used it and I will be honest; it was fast. [T] cozystack -That project took my attention, basically the idea is; it helps you to setup your own cloud. It could be something to spin up quick dev environments. [A] A Distributed Systems Reading List - Want to understand distributed systems better? This article has a list of what you need to read.
A newsletter about DevOps & Machine Learning & Management
[A] The #1 Platform Engineering Problem You've Never Heard Of: Platform Decay - A while ago, I wrote an article with the title No, Platform Engineering Will NOT Do What You Think It Will Do. I still think the same about platform engineering; just adding one more layer to your development process will not magically solve the problems. And, this article/presentation explaining one of the problems in platform engineering: platform decay.
N’s Weekly #59: A newsletter about DevOps & Machine Learning & Management
[T] index_advisor — A PostgreSQL extension for recommending indexes to improve query performance [A] DevOps Road Map — I saw this image and thought someone could use this to be influenced if they are willing to follow a path on DevOps career. [A] How we built Text-to-SQL at Pinterest — This article explains how Pinterest built their Text-to-SQL with LLM, in details. If you are working on a project like that, I am sure this article will provide some quick tips. [A] How Do Machines ‘Grok’ Data?
Hands-On NumPyro: A Practitioner’s Guide to Building Bernoulli Naive Bayes with Confidence Intervals
Naive Bayes is a simple, popular, and straightforward classifier that falls under the category of supervised machine learning. As a data scientist/machine learning engineer, you are likely familiar with the scikit-learn library, which includes an implementation of Bernoulli Naive Bayes. In this article, we will take a step further and explore how to implement Bernoulli Naive Bayes using NumPyro, a probabilistic programming library built on top of JAX.
Apache Spark Optimization Techniques
Large-scale data analysis has become a transformative tool for many industries, with applications that include fraud detection for the banking industry, clinical research for healthcare, and predictive maintenance and quality control for manufacturing. However, processing such vast amounts of data can be a challenge, even with the power of modern computing hardware.
N’s Weekly #57: A newsletter about DevOps & Machine Learning & Management
[T] testkube — A Kubernetes-native testing framework for test execution and orchestration. Even though that project markets itself as a testing framework, I belive with its flow management interface, you can use that tool for more than testing. [T] authelia — A an open source Single Sign-On Multi-Factor portal for web apps. It acts as a companion for reverse proxies by allowing, denying, or redirecting requests.
N’s Weekly #56: A newsletter about DevOps & Machine Learning & Management
[A] The most visited AWS DevOps blog posts in 2023 — AWS published the list of most visited blog posts of 2023. This will give some kind of insight that what was trending in 2023. [A] The Future of DevOps and the Incredible Opportunities that Lie Ahead — An article in which the author shares his opinions about the future of DevOps. I think, at this point, it does not matter which article you read about the future of DevOps — more or less are the same.
N’s Weekly #55: A newsletter about DevOps & Machine Learning & Management
[A] Switching Build Systems, Seamlessly —Here is the story of the migration without stopping development for the more than 120 teams contributing to Spotify’s weekly releases. [T] Tekton — Open source cloud native CI/CD system. You can create CI/CD pipelines on your Kubernetes. [T] Loco — The one-person framework for Rust for side-projects and startups. It seems like a project I would like to play with. [A] In-browser code playgrounds — An article which explains how to run code on browser via WASM.
N’s Weekly #54: A newsletter about DevOps & Machine Learning & Management
[A] A deep dive into CPU requests and limits in Kubernetes — The title article clearly explains what the article is about ;) [V] AWS Re:invent — AWS Re:Invent 2023 was held recently. It is time to visit AWS Events youtube channel to see the latest updates https://www.youtube.com/c/AWSEventsChannel [V] How Google manages open source supply chain security [T] Quarto — An open-source scientific and technical publishing system.
N’s Weekly #53: A newsletter about DevOps & Machine Learning & Management
[A] The Architecture of Serverless Data Systems — The article explains the and advantages of serverless, multi-tenant cloud data services bye explaining in-depth analysis of the design principles and operational challenges of these systems. [A] An intuitive documentation strategy — Writing documentation may not be easy when it comes to software development. Here is an article that may help on the way.
A Case Study using Distil-Whisper and LlaMA2
def speec2text(mp3_file): # Set the computation device to GPU (if available) or CPU device = 'cuda:0' # Choose data type based on CUDA availability (float16 for GPU, float32 for CPU) torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32 # Model identifier for the speech-to-text model model_id = "distil-whisper/distil-large-v2" # Load the model with specified configurations for efficient processing model = AutoModelForSpeechSeq2Seq.from_pretrained( model_id,...
A newsletter about DevOps & Machine Learning & Management
[V] State of DevOps 2023 Highlights [T] OpenFeature - When it comes to software development, if I am the one who leads the process, I prefer to go with trunk based development. And, feature flagging pairs well with this approach. OpenFeature seems like an inititative to standardize the feature flagging [T] depot.ai -Developing machine learning is one thing, deploying it is another. As usual, the common approach is using containers. depot.ai is Docker registry for embedding machine learning models.
A newsletter about DevOps & Machine Learning & Management
Every week (or every 2 weeks), I publish what caught my attention. Even though I am introducing some open source tools that you may have fun using, this week's emphasize is about developer productivity, which you will see the articles below. Enjoy! [T] signoz - Open source Application Performance Monitoring (APM) & Observability tool [T] hoppscotch - Open source Postman Alternative [T] PyTimeTK - A library I came across recently, it may worth to look at it.
A newsletter about DevOps & Machine Learning & Management
[A] The API Gateway and the Future of Cloud Native Applications - In this article the author delves into the evolving role of API gateways in cloud-native applications, emphasizing the importance of standardization and the emerging impact of the Kubernetes Gateway API. [T] SeaGOAT - A code search engine for the AI age. SeaGOAT is a local search tool that leverages vector embeddings to enable to search your codebase semantically.
Artificial Intelligence Development Services
Toptal’s AI development services help businesses improve their decision making and automate business operations and workflows. Top companies and startups work with Toptal AI developers to augment their in-house development teams and build AI applications, software solutions, and machine learning algorithms that solve business problems in areas including customer experience and supply chain. Get Started No-Risk Trial, Pay Only If Satisfied.
A newsletter about DevOps & Machine Learning & Management
[T] open-interpreter - Recently, I keep seeing this tool everywhere. It is a CLI that utilizes GPT API to run code on your terminal on your computer. So, you can just ask; "hey list me aws machines" and it wil list. Well, it is cool. [T] promptfoo - Developing application with LLM prompting can be quite challenging. "Will it behave the same if I change the prompt" is a common question. This tool offers testing for prompts.
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #46, your essential digest on DevOps, Machine Learning, and Management. This week, we delve into the world of Winglang, a programming language for the cloud, and question the effectiveness of DevOps in today's evolving technological landscape. We also introduce Code Llama, Meta's state-of-the-art large language model for coding, and share valuable insights on feature selection tools, generative AI products, and strategies to enhance developer satisfaction and productivity.
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #45,a newsletter about DevOps, Machine Learning, and Management - where I share the articles that caught my eye. This week, we delve into HashiCorp's license change and the consequences it brings, strategies to resolve sporadic timeout issues with AWS Lambda and API Gateway, the eternal debate between frequentist and Bayesian probability, key challenges in Large Language Model (LLM) research, and the urgent need for reskilling in the age of AI and automation.
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #42; I am sharing articles/tools that caught my attention about DevOps, Machine Learning, and Management. This edition explores integration testing for AWS CDK, techniques building LLM-based systems, and insights into large-scale software development.
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #42; I am sharing articles/tools that caught my attention about DevOps, Machine Learning, and Management. This short newsletter will have one great article about establishing an architectural pattern in an organization, one DevOps tool and one Reddit discussion about DevOps practitioners' mistakes, and finally, two tweets in the data science & machine learning section. [A] What are some common mistakes DevOps practitioners make?
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #41; I am sharing articles/tools that took my attention about DevOps, Machine Learning, and Management. We have a short newsletter this week. This week, we explore Makefile's utility in Python automation and cost-saving GPU use with SkyPilot in our DevOps section. We'll see text classification with gzip, a16z's 'companion app,' and expand on Apache Spark's capabilities in the Machine Learning segment.
Vision is the only First Step to Building a Product
A recurrent question echoes through the hallways of offices, boardrooms, and digital conference calls worldwide. A question I've been asked more times than I can count is, "You listened to me. How much will this project cost, and how long will it take?" Surprisingly, my answer is almost always the same: "You told me your vision, but we didn't talk about the product yet." So, why do I respond this way? That's what we will talk about. The cornerstone of any project lies in its vision.
An Exploration of Apache Spark's MLlib
Recently, I kept talking about working with BIG data using Apache Spark: filtering, manipulating, etc .. But, what about machine learning? We will focus on a fascinating component of Spark - MLlib, a library dedicated to Machine Learning. MLlib is one of Spark's main libraries - crafted for machine learning tasks. This library is your ally in breaking down complex machine learning pipelines, making it simpler for us to extract insights from our data.
Yes, Spark Does Speak SQL
We talked about DataFrames and RDDs in Apache Spark in previous articles but can we utilize SQL? The answer is; yes. That's what Apache Spark's SQL module, named Spark SQL, offers. We'll explore how PySpark leverages Spark SQL and why this integration should matter to you. Spark SQL lets you work with structured data, and you can mix SQL queries right into Spark programs. It doesn't matter if you're coding in Java, Scala, Python, or R, it works with all of them.
Exploring Filtering, Mapping, and Reducing
In this article, we'll be discussing some fundamental data processing tasks you can accomplish using PySpark - namely, filtering, mapping, and reducing. By the end of this post, you'll have a understanding of how these tasks work and how to apply them to your PySpark operations. Before we dive into the details, let's remind ourselves of the basics from the previous article.
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #39; I am sharing articles/tools that took my attention about DevOps, Machine Learning, and Management. This week, we explore the latest SRE report, the utility of DevPod, AI vulnerabilities, the role of Large Language Models in autonomous agents, Evidently Python library, and the impact of a listening culture in leadership. Let's dive in! [A] 2023 SRE Report - It is worth taking a look.
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #38, a newsletter in that I share what took my attention about DevOps, Machine Learning, and Management. This edition discusses cybersecurity risks in ChatGPT, managing dev environments with Nix, FinGPT for finance, interactive data exploration with "weave," balanced leadership, and building large technical projects. Enjoy the read! [A] Can you trust ChatGPT's package recommendations?
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #37, a newsletter about DevOps, Machine Learning, and Management. In this edition, discover the benefits of trunk-based development, tackle cloud drift with Policy-as-Code, explore tokens in NLP, dive into causal inference, and uncover data-driven decision-making. [A] Why I prefer trunk-based development -I prefer trunk-based development. Here is an article from an author who shares the same idea.
A Simplified Guide
Natural Language Processing (NLP) has revolutionized how we interact with machines, enabling them to understand, interpret, generate, and do much more with human language. A fundamental concept that makes this possible is the concept of a 'token.' Understanding what tokens are and their role in NLP is essential to understanding how these language models work. In this blog post, we'll explain the concept of tokens and delve into their role in the context of large language models (LLMs) like GPT.
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #36; I am sharing articles that take my attention every week about DevOps & Machine Learning & Management. This edition features Kestra, a robust orchestration platform, and an in-depth look at AI vulnerabilities in our DevOps & Tools section. For Data Science & Machine Learning, we provide a handy list of interview questions, introduce the Falcon Models under Apache 2.0 License, and offer a simplified guide to understanding text embeddings.
A Simplified Guide
Introduction to Text Embeddings Text embeddings are crucial to modern machine learning, especially in natural language processing (NLP). But what exactly are they? Simply put, text embeddings are a method of translating words or sentences from human language into the language of computers - numbers. This numerical representation of text allows machines to understand and process language meaningfully, enabling many of the advanced NLP applications we use today.
A newsletter about DevOps & Machine Learning & Management
Welcome to N's Weekly #35! This newsletter covers DevOps, Machine Learning, and Management. In DevOps & Tools, we discuss the lessons learned from Amazon's microservices to monolith story and explore Martin Fowler's investigation into using ChatGPT in software development. In Data Science & Machine Learning, we share intriguing tools like privateGPT for private document interaction and CLI tools llm, ttok, and strip-tags for working with ChatGPT.
A Personal Story of How One Small Change...
Introduction In today's fast-paced world, effective communication is more critical than ever, especially for technical teams. A common issue many tech professionals face is the struggle to convey their progress and achievements to stakeholders, which can lead to misperceptions about the team's efficiency and capabilities.
You can't motivate your team
While scrolling through my Twitter timeline, I saw the following tweet from Allen Holub, and I wanted to share my two cents about that topic. It's a common belief that a manager's role is to motivate people, but it's not entirely true. While you may not directly motivate people, you can create an environment where people motivate themselves. As Lee Cockerell said; As a leader, you want to inspire your team.
N's Weekly #19
A newsletter about DevOps & Machine Learning & Management [A] AWS re:Invent - AWS re:Invent finished recently and you can see all the keynotes and the sessions on their webpage: https://reinvent.awsevents.com/on-demand/ [T] localstack - A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline! [A] From Service to Platform: A Ranking System in Go [A] ChatGPT: Optimizing Language Models for Dialogue - ChatGPT is storming the internet for the last 10 days.
N's Weekly #7
[T] kratix - Kratix is a framework for building Platform-as-a-Product [A] Creating Automation Credentials Without Exposing Them To Users - A set of posts that explain how to create automation credentials on AWS without exposing them to the users [T] CRDs-catalog - Over 100 popular Kubernetes CRDs (CustomResourceDefinition) in JSON schema format.
Machine learning does not produce value for my business. Why?
By Necati Demir, Co-Founder & CTO at datagran.io Anton Grabolle / Better Images of AI / Classification Cupboard / Licenced by CC-BY 4.0 The keyword “machine learning” is around all the time. You keep hearing it and people tell it like magic that solves everything, and as a leader you want to solve your problems in your business.
Ensemble Methods: Elegant Techniques to Produce Improved Machine Learning Results
By Necati Demir. Ensemble methods are techniques that create multiple models and then combine them to produce improved results. Ensemble methods usually produces more accurate solutions than a single model would. This has been the case in a number of machine learning competitions, where the winning solutions used ensemble methods. In the popular Netflix Competition, the winner used an ensemble method to implement a powerful collaborative filtering algorithm.
Ensemble Methods: Elegant Techniques to Produce Improved Machine Learning Results
Ensemble methods are techniques that create multiple models and then combine them to produce improved results. Ensemble methods usually produces more accurate solutions than a single model would. This has been the case in a number of machine learning competitions, where the winning solutions used ensemble methods. In the popular Netflix Competition, the winner used an ensemble method to implement a powerful collaborative filtering algorithm.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Necati
Contact Necati, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack