skeptric - Skeptric

Dataset of 2024 Victorian Council Election Candidates

data

In local elections what the candidates campaign on tells us a lot about what matters to them, and by extension to their communities. To understand what is…

Measuring a Language Model

nlp

makemore

Implementing Lempel-Ziv 77 Paper in Python

entropy

Lossless compression is the process of taking some data and squishing it into a smaller space without losing any information. It is…

Lossless Compression with Huffman Codes

entropy

Lossless compression algorithms are almost magical; you can take some data source squish it down into a smaller space, and then restore it back to its full size later. They…

Stanford AI Professional Program Review

In 2023 I completed the Stanford AI Professional Program to deepen my understanding of Artificial Intelligence, especially with natural language. The courses I took were great and…

Mamba Lanugage Model Inference

nlp

llm

Mamba is an alternative kind of neural network model to Transformers inspired by State Space Models. It can train as efficiently as…

A dataset of P. G. Wodehouse books

gutenberg

We are going to create a text dataset from the books of P. G. Wodehouse for language modelling. I enjoy his style of writing…

Roman Numerals with Python and Regular Expressions

python

We’re going convert integers to roman numerals in standard form and back again in Python, as well as detect them with regular expressions.

Downloading books from Project Gutenberg with Python

nlp

gutenberg

Project Gutenberg is a…

Makemore Subreddits - Part 4 Backprop

nlp

makemore

Let’s manually backpropagate through in a Multi-Layer Perceptron language model for subreddit names.

Makemore Subreddits - Part 3 Activations and Gradients

nlp

makemore

Let’s dive deep into the Activations and Gradients in a Multi-Layer Perceptron language model for subreddit names.

Building a Deep Learning Computer for AU$2000

computing

nlp

Building a reliable Deep Learning workstation for around AU$2,000 is achievable even for beginners. In my case, I constructed a PC with a used RTX 3090, a 6-core 3.9GHz CPU…

Starting AWS EC2 Compute Instances from the Command Line

aws

The ability to rent compute resources on demand is incredibly useful as a data scientist. There are often batch processing jobs, such as training a machine learning model…

Makemore Subreddits - Part 2 Multilayer Perceptron

nlp

makemore

Let’s create a Multi-Layer Perceptron language model for subreddit names.

Makemore Subreddits - Part 1 Bigram Model

nlp

makemore

Let’s create a bigram character level language model for subreddit names. This comes at a challenging time when Reddit is changing the terms of its API making most third party apps infeasible (and data extracts like this analysis relies on prohibitively expensive). However the Reddit community has been an interesting place…

Centroid Spherical Polygon

maths

You’re organising a conference of operations research analysts from all over the world, but their…

Thumbs Up? Sentiment Classification Like it’s 2002

nlp

sentiment

In July 2002 Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan published Thumbs up? Sentiment Classification using Machine Learning Techniques. at EMNLP, one of the earliest works of using machine learning for Sentiment…

Learning Natural Language Processing through Sentiment Classification

nlp

sentiment

There is currently a lot of hype in Natural Language Processing, much of it driven by Open AI’s effective marketing of its GPT systems. As someone who has only been working…

Linear Stacking Cosine Embeddings

Why pretrain your own language model?

nlp

I think now is a great time to pretrain your own language model from scratch. This may be a strange statement when all the best performing models today are hundreds of…

How not to Evaluate NER Systems

nlp

hnbooks

ner

A typical way to evaluate NER (Named Entity Recognition) Systems is to look at the F1 score, however this is a bad idea as stated in Chris Manning’s 2006 blog post Doing Named Entity Recognition? Don’t…

Conda Environment YAML for running TensorFlow on GPU

python

Getting TensorFlow to work on a GPU can be tricky, but conda can make it relatively easy. Here’s a configuration that I find works on TensorFlow 2.11 with CUDA 11.7:

Convert Hugo mmark LaTeX into Pandoc

blog

I’ve recently migrated from Hugo to Quarto and one of the hardest steps was converting the equations in Hugo’s legacy mmark format to Quarto. This notebook shows how I converted the…

Sinusoidal Functions as Harmonic Oscillations

maths

The sinusoidal functions (like sin and cosine) normally first come up when students are learning about angles of right angled triangles. However there are many…

Migrating from Hugo to Quarto

blog

I’ve just moved this website from Hugo to Quarto, and I am very happy. Quarto is much better for a mathematics, data, and…