Killing the LSTM: Why DANI is Pivoting to a Spiking Neural Network

After some extensive research and a few hard encounters with hardware reality, I’ve made a major architectural decision regarding DANI's brain: I am completely ripping out the LSTM neural network and replacing it with a Recurrent Spiking Neural Network (RSNN).

Given my recent enthusiasm for the LSTM architecture, this will probably come as a surprise to anyone following along. However, I have my reasons, and as usual, they come down to math, metal, and thermal throttling.

What is wrong with LSTMs and the Traditional ANN?

Traditional Artificial Neural Networks (ANNs) are fantastic at what they were designed for: probability scoring. They allow us to simulate decision-making using massive matrix calculations mixed with a healthy dose of calculus.

But beneath the hood, they are brute-force mechanisms.

When a traditional ANN is fed data, that data cascades through every single neuron in the network. Even when a neuron has a zero value, the CPU still dutifully performs the mathematical operation of multiplying by zero. If a network has a shape of 10 inputs, 10 outputs, and 5 layers of 12 neurons each, that is 816 floating-point calculations just for one single feed-forward pass. If you start adding in simple recurrence, we are immediately up to 1,488 calculations.

Now, scale this up to something that would actually be useful for DANI. We need at least 100 inputs and about 64 outputs. With hidden layers of around 125 neurons each, we jump to 150,375 calculations per pass. To achieve an LSTM architecture, we can estimate the computation to be roughly four times that: over 600,000 calculations per tick.

As you can see, this grows exponentially. More importantly, it keeps the CPU running permanently hot. Don’t forget, DANI is powered by a Raspberry Pi 5. There is no GPU offloading here. I did manage to get DANI’s LSTM operating at 10Hz with 3.5 million parameters, but it required the Pi 5 to do absolutely nothing else, running flat-out at 100% utilization. DANI's brain was essentially doubling as a space heater, even when he was just "dreaming."

Once you add my simulated hormone system into the mix, the compute overhead goes up even further. I was basically following the same brute-force model that the big AI companies use, just without the luxury of a multi-million dollar server farm.

And then there is the training. To train an ANN of this style requires layer-by-layer calculus (backpropagation). This is heavy, blocking work. It takes a considerable amount of time, meaning we simply cannot train the network in real-time on anything substantial.

The Elegance of the Recurrent Spiking Neural Network

The architecture of an RSNN is topographically similar to other neural networks, but the execution is fundamentally different. It mimics biological reality much closer.

Instead of passing continuous floating-point numbers, each neuron has a membrane potential. As it receives signals from upstream neurons, that potential increases (or decreases, if the synapse is inhibitory). The neuron does absolutely nothing until that potential reaches a specific threshold. Once it hits the limit, it fires—or "spikes." And it spikes at a binary full power. The neuron then resets, either dropping to zero or subtracting the threshold from its current potential.

If a neuron doesn’t reach the threshold, it doesn’t spike. Period.

This means we only need to calculate the pathways for neurons that actually receive a spike. The computational reduction is staggering. The general consensus in the field is that for any given signal, only 5% to 10% of a spiking network is active, compared to the 100% density of a matrix-style network.

Training also becomes radically simpler. We only need to reinforce or weaken the synapses of the pathways that actually fired (a process akin to Spike-Timing-Dependent Plasticity). We apply a localized effector to those specific connections—no massive, network-wide calculus required.

But here is the real kicker for a systems engineer: an RSNN allows us to completely ditch floating-point math. By using integer mathematics and clever bit-shift operations (which are essentially free in terms of CPU cycles), we can completely bypass the messy, cycle-heavy floating-point multiplications.

How does this help DANI?

Obviously, DANI will not have to run so hot. If he has a quiet mind, the network will physically go quiet. When DANI rests, he will literally be saving power. By utilizing temporal calculations, we can also train him in real-time with near-zero impact on the system.

Furthermore, DANI’s brain becomes an event-driven architecture rather than a strict polling loop. Instead of processing rigidly on a timer, he will think and react in time with the world around him.

I will still be using the hormonal system I designed, but the key difference is how it integrates. The effective values of the hormones will now directly modulate the firing thresholds of the neurons. This allows his "moods" to physically alter the state of his brain and affect his behavior, even if his underlying memories remain unchanged.

I suppose I had better be nice to him.

Rethinking Asimov’s Laws

This pivot finally allows me to revisit a question I was pondering a while back: How do we encode Asimov’s Laws into an AI?

The answer is: we don’t. It’s as simple as that. We don’t hardcode the laws; we teach them.

By running an RSNN that learns in real-time through stimulus and reinforcement, the laws can become an emergent part of his personality. He will understand that hurting people is bad and that following human orders brings a reward, not because a line of code forces him to, but because his synapses have shaped themselves around those experiences.

So, I guess I really am going to have to be a parent to a robot.

Major Milestone: Skink Has Officially Bootstrapped!

Woohoo! It’s time for a long-overdue update on the language powering the future of DANI.

We just crossed a monumental threshold with Skink: it is now officially bootstrapped!

For anyone unfamiliar with compiler development, bootstrapping means that the Skink compiler can now compile its own source code. From here on out, all future versions of the Skink compiler will be written in Skink itself. It’s a huge rite of passage for any programming language project, and seeing it actually work after so much foundation work is insanely satisfying.

Where Go Fits In (For Now)

The handover

Now, does this mean I’m throwing Go out the window today? Not quite!

Go has a proven, rock-solid track record, and I’ll be keeping it around for prototyping work. Whenever I want to test out an idea quickly and I’m not sure if Skink is quite ready to handle the heavy lifting yet, Go gives me a reliable safety net.

However, the long-term vision hasn't changed: eventually, I plan to fully commit to Skink for almost all development across my projects, experiments, and research.

The Road to Skink 1.0

While bootstrapping is a massive step forward, we aren't quite at version 1.0 just yet. To earn that release tag, Skink needs to hit a few crucial targets:

1. Cross-Platform Support: Clean cross-compilation across Windows, macOS, and Linux.
2. Architecture Support: Full targeting for both x86 and ARM architectures.
3. Embedded Target Transpilation: The ability to compile (or at least transpile down) to low-power platforms like the K210, ESP32, and standard Arduino C/C++.
4. Direct Hardware I/O: First-class hardware abstraction libraries for lower-level protocols—specifically I2C, SPI, GPIO, and related interfaces.

Acceleration, Compute, and Graphics Backends

Beyond microcontrollers and standard CPUs, getting high-performance compute and neural inference support hooked up is high on the priority list.

• CUDA Support: I still need to get CUDA fully enabled and tested. Because of my current local setup, this will likely require either acquiring new dedicated NVIDIA hardware or setting up a cloud-based VM for build and test pipelines.
• 
  Vulkan & AMD: My main local system currently relies on Vulkan to run LM Studio, so a Vulkan compute backend is high on the radar as a viable cross-vendor path. I'm also looking closely at AMD’s libraries (ROCm/HIP) to ensure broad hardware compatibility.

It's a lot of moving parts, but watching the architecture take shape piece by piece is incredibly rewarding.

Bringing It Back to DANI

So, why go through all the trouble of building a custom language from scratch?

It all comes down to DANI. My goal has always been to have a single, unified language capable of bridging every layer of DANI’s stack—from high-level logic and neural compute all the way down to real-time bare-metal sensor and actuator control. Skink is the key to making that happen without juggling three different language ecosystems.

Speaking of DANI, there have also been some fascinating developments regarding his neural network architecture recently... but I’ll leave you hanging on that for now and save the deep dive for the next post!

Stay tuned!

Oops, I Gave My Robot Amnesia (And How I'm Fixing It)

Wow, it’s been a while. Apologies for the radio silence, but the pesky "real world" caught up with me, and I had to spend some time doing that whole "working for a living" thing.

Anyway, enough about the mundane. Let's get back to what is actually important: DANI.

As you might remember, my ultimate, beyond-my-wildest-dreams goal with this project is to cross that threshold and meet the definition of when a robot is actually alive, or at least close to it. But recently, while pondering DANI’s LSTM (the fancy Long Short-Term Memory neural network that acts as his brain), I realized I had made a fundamental—and slightly embarrassing—mistake.

It’s hard to achieve sentience when your robot has the memory retention of a goldfish.

The Problem: Scheduled Blackouts

As it stands right now, DANI "thinks" every 100 milliseconds, giving him 10 thought cycles a second. Every 10 seconds (100 cycles), backpropagation kicks in to train the network. To do this concurrently without stopping DANI in his tracks, I clone the LSTM at that exact moment, run the heavy backpropagation math on the clone, and then overwrite the active LSTM with the newly trained clone.

This backpropagation takes about 2 to 3 seconds. My initial thought was: Brilliant! The training happens in the background without interrupting his flow.

But there is a glaring flaw.

Because the process takes a snapshot, spends 3 seconds learning from it, and then violently overwrites the active brain... we lose those 2 to 3 seconds of short-term memory that DANI experienced while the training was happening. Every 10 seconds, DANI essentially blacks out and forgets the last few seconds of his existence. This is seriously hampering his learning capabilities.

How do we stop DANI from becoming a chronic amnesiac?

The Fix: A Neurological Hot-Swap

My solution is to ditch the cloning process entirely. Instead, each neuron will now have two sets of weights: one active, one inactive.

During backpropagation, the inactive weights will get the results of the calculation (using the active weights for the algorithm). This allows us to update the LSTM's underlying math without wiping out the actively evolving memory states (the cell states and hidden states) that DANI is currently using to understand the world. We just add a flag to each layer to indicate whether it should be reading from Weight Set 1 or Weight Set 2.

But wait, there’s more!

Reshaping the Brain

At present, DANI's model has about 300 neurons on each layer, with 5 layers (I don’t have the code right in front of me, so I'm relying on my own somewhat flawed, non-LSTM memory here).

If we increase the number of layers but reduce the neurons per layer, we can implement a rolling update. This means DANI can immediately benefit from the training layer-by-layer, even while the rest of the brain is still calculating.

What this entails is increasing the layer count to 7 (any higher and we start flirting with the dreaded vanishing gradient problem), but reducing the neuron count, per layer, to 128 (because who doesn't love a nice power of 2?).

This gives DANI a much more focused, "deep" thought process, allowing him to break down problems more efficiently. It also allows us to gracefully ‘flip the switch’ on each layer as we cycle through.

Here is how the rolling update will work:

As each feed-forward pass occurs (DANI thinking), a check is done to see if the next layer is ready to have its switch flipped to the newly trained weights. Because backpropagation is strictly sequential and works backwards, we start checking from the last layer and move towards the first.

If a layer is ready, we flip the weights to the newly trained set and mark it as done. On the next thought cycle, we check the next layer, and so on, until we reach the front of the brain. Then, we start the whole process over again.

What do we gain from this brain surgery?

Quite a bit, actually:

1. Constant Learning: The LSTM is in a state of continuous, uninterrupted learning.
2. Stable Learning Rate: No massive, sudden shifts in logic.
3. Smoother Processing: No sudden CPU spikes from cloning and overwriting massive arrays.
4. Deeper Thinking: The structural change to 7 layers gives DANI a more nuanced, layered way of processing information.
5. Memory Retention: We actually retain the states of the memory gates within the LSTM. No more 3-second blackouts!

There are certainly other ways to create a continuous neural network, but I am aiming for the absolute simplest solution here. Remember, all of this is running on a Raspberry Pi!

This dual-weight method does increase the memory required to hold the LSTM, but because we are reducing the overall neuron count from ~1500 (5x300) to 896 (7x128), it's actually going to be lighter on the Pi overall. DANI had an oversized network anyway, so trimming the fat while adding depth is a win-win.

What do you guys think of this approach? Let me know in the comments if you see any potholes I'm about to step in!

Vectorizing Memory

Hello, fellow explorers of the digital frontier!

You know how it is when you're building an AI, especially one destined for the real world, embodied in a robot head (and maybe a mobile platform, wink wink)? You need a brain, and that brain needs a memory. But not just any memory – it needs a memory that understands meaning, not just keywords. And that, my friends, is where the humble, yet mighty, Vector Database comes into play.

For those of you following my DANI project, you'll know I'm all about pushing intelligence to the edge, directly onto Single Board Computers (SBCs) like our beloved Raspberry Pis. This week, I want to dive into why vector databases are absolutely crucial for this vision, and how I'm tackling the challenge of making them lightweight enough for our resource-constrained little friends.

What in the World is a Vector Database, Anyway?

Forget your traditional spreadsheets and relational tables for a moment. A vector database is a special kind of database built from the ground up to store, index, and query vector embeddings efficiently. Think of these embeddings as multi-dimensional numerical representations of anything unstructured: text, images, audio, even your cat's purr. The magic? Semantically similar items are positioned closer to each other in this high-dimensional space.

Unlike a traditional database that looks for exact matches (like finding "apple" in a list), a vector database looks for similar meanings (like finding "fruit" when you search for "apple"). This is absolutely foundational for modern AI, especially with the rise of Large Language Models (LLMs). Vector databases give LLMs a "memory" beyond their training data, allowing them to pull in real-time or proprietary information to avoid those pesky "hallucinations" and give us truly relevant answers.

The process involves: Embedding (turning your data into a vector using an AI model), Indexing (organizing these vectors for fast searching, often using clever Approximate Nearest Neighbor (ANN) algorithms like HNSW or IVF), and Querying (finding the "closest" vectors using metrics like Cosine Similarity). It's all about finding the semantic buddies in a vast sea of data!

SBCs: The Tiny Titans of the Edge

Now, here's the rub. While big cloud servers can throw endless CPU and RAM at vector databases, our beloved SBCs (like the Raspberry Pi) are a bit more... frugal. They have limited CPU power, often less RAM than your phone, and slower storage (those pesky microSD cards!). This creates what I call the "Accuracy-Speed-Memory Trilemma." You can have two, but rarely all three, without some serious wizardry.

For my DANI project, the goal is to have intelligence on the device, reducing reliance on constant cloud connectivity. This means our vector database needs to be incredibly lightweight and efficient. Running a full-blown client-server database daemon just isn't going to cut it.

My Go-To for Go: github.com/trustingasc/vector-db

This is where the Go ecosystem shines for embedded systems. While there are powerful vector databases like Milvus or Qdrant, their full versions are too heavy. What we need is an embedded solution – something that runs as a library within our application's process, cutting out all that pesky network latency and inter-process communication overhead.

My current favourite for this is github.com/trustingasc/vector-db. It's a pure Go-native package designed for efficient similarity search. It supports common distance measures like Cosine Similarity (perfect for semantic search!) and aims for logarithmic time search performance. Being Go-native means seamless integration and leveraging Go's fantastic concurrency model.

Here's a simplified peek at how we'd get it going in Go (no calculus required, I promise!):

package main

import (
  "fmt"
  "log"
  "github.com/trustingasc/vector-db/pkg/index"
)

func main() {
  numberOfDimensions := 2 // Keep it simple for now!
  distanceMeasure := index.NewCosineDistanceMeasure()
  vecDB, err := index.NewVectorIndex[string](2, numberOfDimensions, 
    5, nil, distanceMeasure)
  if err != nil { log.Fatalf("Failed to init DB: %v", err) }
  fmt.Println("Vector database initialized!")
  // Add some data points (your AI's memories!)
  vecDB.AddDataPoint(index.NewDataPoint("hello", []float64{0.1, 0.9}))
  vecDB.AddDataPoint(index.NewDataPoint("world", []float64{0.05, 0.85}))
  vecDB.Build()
  fmt.Println("Index built!")
  // Now, search for similar memories!
  queryVector := []float64{0.12, 0.92}
  results, err := vecDB.SearchByVector(queryVector, 1, 1.0)
  if err != nil { log.Fatalf("Search error: %v", err) }
  for _, res := range *results {
    fmt.Printf("Found: %s (Distance: %.4f)\n", res.ID, res.Distance)
  }
}

This little snippet shows the core operations: initializing the database, adding your AI's "memories" (vector embeddings), and then searching for the most similar ones. Simple, elegant, and perfect for keeping DANI's brain sharp!

Optimizing for Tiny Brains: The Trilemma is Real!

The "Accuracy-Speed-Memory Trilemma" is our constant companion on SBCs. We can't just pick the fastest or most accurate index; we have to pick one that fits. This often means making strategic compromises:

Indexing Algorithms: While HNSW is great for speed and recall, it's a memory hog. For truly constrained environments, techniques like Product Quantization (PQ) are game-changers. They compress vectors into smaller codes, drastically reducing memory usage, even if it means a tiny trade-off in accuracy. It's about getting the most bang for our limited memory buck!

Memory Management: Beyond compression, we're looking at things like careful in-memory caching for "hot" data and reducing dimensionality (e.g., using PCA) to make vectors smaller. Every byte counts!

Data Persistence: MicroSD cards are convenient, but they're slow and have limited write endurance. For embedded Go libraries, this means carefully serializing our index or raw data to disk and loading it on startup. We want to avoid constant writes that could wear out our precious storage.

It's a constant dance between performance and practicality, ensuring DANI can learn and remember without needing a supercomputer in its head.

The Road Ahead: Intelligent Edge and DANI's Future

Vector databases are more than just a cool piece of tech; they're foundational for the kind of intelligent, autonomous edge applications I'm building with DANI. By enabling local vector generation and similarity search, we can power real-time, context-aware AI without constant reliance on the cloud. Imagine DANI performing on-device anomaly detection, localized recommendations, or processing commands without a hiccup, even if the internet decides to take a nap!

This journey is all about pushing the boundaries of what's possible with limited resources, making AI smarter and more independent. It's challenging, exciting, and occasionally involves me talking to a Raspberry Pi as if it understands me (it probably does, actually).

What are your thoughts on running advanced AI components on tiny machines? Have you dabbled in vector databases or edge computing? Let me know in the comments below!