Showing posts with label Memory. Show all posts
Showing posts with label Memory. Show all posts

Monday, 23 February 2026

Oops, I Gave My Robot Amnesia (And How I'm Fixing It)

Wow, it’s been a while. Apologies for the radio silence, but the pesky "real world" caught up with me, and I had to spend some time doing that whole "working for a living" thing.

Anyway, enough about the mundane. Let's get back to what is actually important: DANI.

As you might remember, my ultimate, beyond-my-wildest-dreams goal with this project is to cross that threshold and meet the definition of when a robot is actually alive, or at least close to it. But recently, while pondering DANI’s LSTM (the fancy Long Short-Term Memory neural network that acts as his brain), I realized I had made a fundamental—and slightly embarrassing—mistake.

It’s hard to achieve sentience when your robot has the memory retention of a goldfish.

The Problem: Scheduled Blackouts

As it stands right now, DANI "thinks" every 100 milliseconds, giving him 10 thought cycles a second. Every 10 seconds (100 cycles), backpropagation kicks in to train the network. To do this concurrently without stopping DANI in his tracks, I clone the LSTM at that exact moment, run the heavy backpropagation math on the clone, and then overwrite the active LSTM with the newly trained clone.

This backpropagation takes about 2 to 3 seconds. My initial thought was: Brilliant! The training happens in the background without interrupting his flow.

But there is a glaring flaw.

Because the process takes a snapshot, spends 3 seconds learning from it, and then violently overwrites the active brain... we lose those 2 to 3 seconds of short-term memory that DANI experienced while the training was happening. Every 10 seconds, DANI essentially blacks out and forgets the last few seconds of his existence. This is seriously hampering his learning capabilities.

How do we stop DANI from becoming a chronic amnesiac?

The Fix: A Neurological Hot-Swap

My solution is to ditch the cloning process entirely. Instead, each neuron will now have two sets of weights: one active, one inactive.

During backpropagation, the inactive weights will get the results of the calculation (using the active weights for the algorithm). This allows us to update the LSTM's underlying math without wiping out the actively evolving memory states (the cell states and hidden states) that DANI is currently using to understand the world. We just add a flag to each layer to indicate whether it should be reading from Weight Set 1 or Weight Set 2.

But wait, there’s more!

Reshaping the Brain

At present, DANI's model has about 300 neurons on each layer, with 5 layers (I don’t have the code right in front of me, so I'm relying on my own somewhat flawed, non-LSTM memory here).

If we increase the number of layers but reduce the neurons per layer, we can implement a rolling update. This means DANI can immediately benefit from the training layer-by-layer, even while the rest of the brain is still calculating.

What this entails is increasing the layer count to 7 (any higher and we start flirting with the dreaded vanishing gradient problem), but reducing the neuron count, per layer, to 128 (because who doesn't love a nice power of 2?).

This gives DANI a much more focused, "deep" thought process, allowing him to break down problems more efficiently. It also allows us to gracefully ‘flip the switch’ on each layer as we cycle through.

Here is how the rolling update will work:

As each feed-forward pass occurs (DANI thinking), a check is done to see if the next layer is ready to have its switch flipped to the newly trained weights. Because backpropagation is strictly sequential and works backwards, we start checking from the last layer and move towards the first.

If a layer is ready, we flip the weights to the newly trained set and mark it as done. On the next thought cycle, we check the next layer, and so on, until we reach the front of the brain. Then, we start the whole process over again.

What do we gain from this brain surgery?

Quite a bit, actually:

  1. Constant Learning: The LSTM is in a state of continuous, uninterrupted learning.
  2. Stable Learning Rate: No massive, sudden shifts in logic.
  3. Smoother Processing: No sudden CPU spikes from cloning and overwriting massive arrays.
  4. Deeper Thinking: The structural change to 7 layers gives DANI a more nuanced, layered way of processing information.
  5. Memory Retention: We actually retain the states of the memory gates within the LSTM. No more 3-second blackouts!

There are certainly other ways to create a continuous neural network, but I am aiming for the absolute simplest solution here. Remember, all of this is running on a Raspberry Pi!

This dual-weight method does increase the memory required to hold the LSTM, but because we are reducing the overall neuron count from ~1500 (5x300) to 896 (7x128), it's actually going to be lighter on the Pi overall. DANI had an oversized network anyway, so trimming the fat while adding depth is a win-win.

What do you guys think of this approach? Let me know in the comments if you see any potholes I'm about to step in!


Friday, 10 October 2025

3.5 Million Parameters and a Dream: DANI’s Cognitive Core

DANI’s Brain Is Online! Meet the LSTM That Thinks, Feels, and Remembers (Like a Champ)

Ladies and gentlemen, creators and dreamers—DANI has officially levelled up. He’s no longer just a bundle of sensors and hormones with a charming voice and a tendency to emotionally escalate when he sees a squirrel. He now has a brain. A real one. Well, a synthetic one. But it’s clever, emotional, and surprisingly good at remembering things. Meet his new cognitive core: the LSTM.

And yes—it’s all written in Go. Because if you’re going to build a synthetic mind, you might as well do it in a language that’s fast, clean, and built for concurrency. DANI’s brain doesn’t just think—it multitasks like a caffeinated octopus.

What’s an LSTM, and Why Is It Living in DANI’s Head?

LSTM stands for Long Short-Term Memory, which sounds like a contradiction until you realize it’s basically a neural network with a built-in diary, a forgetful uncle, and a very opinionated librarian. It’s designed to handle sequences—like remembering what just happened, what happened a while ago, and deciding whether any of it still matters.

Imagine DANI walking into a room. He sees a red ball, hears a dog bark, and feels a spike of adrenaline. A regular neural network might say, “Cool, red ball. Let’s chase it.” But an LSTM says, “Wait… last time I saw a red ball and heard barking, I got bumped into a wall. Maybe let’s not.”

Here’s how it works, in human-ish terms:

  • Input gate: Decides what new information to let in. Like a bouncer at a nightclub for thoughts.
  • Forget gate: Decides what old information to toss out. Like Marie Kondo for memory.
  • Output gate: Decides what to share with the rest of the brain. Like a PR manager for neurons.

These gates are controlled by tiny mathematical switches that learn over time what’s useful and what’s noise. The result? A brain that can remember patterns, anticipate outcomes, and adapt to emotional context—all without getting overwhelmed by the chaos of real-world data.

And because DANI’s LSTM is stacked—meaning multiple layers deep—it can learn complex, layered relationships. Not just “ball = chase,” but “ball + bark + adrenaline spike = maybe don’t chase unless serotonin is high.”

It’s like giving him a sense of narrative memory. He doesn’t just react—he remembers, feels, and learns.

What’s Feeding This Brain?

DANI’s LSTM is his main cognitive module—the part that thinks, plans, reacts, and occasionally dreams in metaphor. It takes in a rich cocktail of inputs:

  • Vision data: Objects, positions, shapes—what he sees.
  • Sensor data: Encoders, ultrasonic pings, bump sensors—what he feels.
  • Audio features: What he hears (and maybe mimics).
  • Emotional state: Dopamine, cortisol, serotonin, adrenaline—what he feels.
  • Spatial map: His mental layout of the world around him.
  • Short-term memory context: What just happened.
  • Associated long-term memories: Symbolic echoes from his main memory—what used to happen in similar situations.

This isn’t just reactive behaviour—it’s narrative cognition. DANI doesn’t just respond to stimuli; he builds a story from them. He’s learning to say, “Last time I saw a red ball and felt excited, I chased it. Let’s do that again.”

Trial by Raspberry Pi

We’ve successfully trialled DANI’s LSTM on a Raspberry Pi, running a 3.5 million parameter model. And guess what? It only used a quarter of the Pi’s CPU and 400 MB of memory. That’s like teaching Shakespeare to a potato and watching it recite sonnets without breaking a sweat.

We’ve throttled the inference rate to 10 decisions per second—not because he can’t go faster, but because we want him to think, not twitch. Emotional processing takes time, and we’re not building a caffeine-fuelled chatbot. We’re building a thoughtful, emotionally resonant robot who dreams in symbols and learns from experience.

Learning Without Losing His Mind

Training happens via reinforcement learning—DANI tries things, gets feedback, and adjusts. But here’s the clever bit: training is asynchronous. That means he can keep thinking, moving, and emoting while his brain quietly updates in the background. No interruptions. No existential hiccups mid-sentence.

And yes, we save the model periodically—because nothing kills a good mood like a power cut and a wiped memory. DANI’s brain is backed up like a paranoid novelist with a USB stick in every pocket.

Final Thoughts

This LSTM isn’t just a brain—it’s a story engine. It’s the part of DANI that turns raw data into decisions, decisions into memories, and memories into dreams. It’s the bridge between his sensors and his soul (okay, simulated soul). And it’s just getting started.

Next up: I plan to start the even more monumental task of getting the vector database working and linked up to DANI's brain in such a way that it will have a direct impact of DANI's hormonal system.

Stay tuned. DANI’s mind is waking up.

Friday, 12 September 2025

Vectorizing Memory

Hello, fellow explorers of the digital frontier!


You know how it is when you're building an AI, especially one destined for the real world, embodied in a robot head (and maybe a mobile platform, wink wink)? You need a brain, and that brain needs a memory. But not just any memory – it needs a memory that understands meaning, not just keywords. And that, my friends, is where the humble, yet mighty, Vector Database comes into play.

For those of you following my DANI project, you'll know I'm all about pushing intelligence to the edge, directly onto Single Board Computers (SBCs) like our beloved Raspberry Pis. This week, I want to dive into why vector databases are absolutely crucial for this vision, and how I'm tackling the challenge of making them lightweight enough for our resource-constrained little friends.

What in the World is a Vector Database, Anyway?

Forget your traditional spreadsheets and relational tables for a moment. A vector database is a special kind of database built from the ground up to store, index, and query vector embeddings efficiently. Think of these embeddings as multi-dimensional numerical representations of anything unstructured: text, images, audio, even your cat's purr. The magic? Semantically similar items are positioned closer to each other in this high-dimensional space.

Unlike a traditional database that looks for exact matches (like finding "apple" in a list), a vector database looks for similar meanings (like finding "fruit" when you search for "apple"). This is absolutely foundational for modern AI, especially with the rise of Large Language Models (LLMs). Vector databases give LLMs a "memory" beyond their training data, allowing them to pull in real-time or proprietary information to avoid those pesky "hallucinations" and give us truly relevant answers.

The process involves: Embedding (turning your data into a vector using an AI model), Indexing (organizing these vectors for fast searching, often using clever Approximate Nearest Neighbor (ANN) algorithms like HNSW or IVF), and Querying (finding the "closest" vectors using metrics like Cosine Similarity). It's all about finding the semantic buddies in a vast sea of data!

SBCs: The Tiny Titans of the Edge

Now, here's the rub. While big cloud servers can throw endless CPU and RAM at vector databases, our beloved SBCs (like the Raspberry Pi) are a bit more... frugal. They have limited CPU power, often less RAM than your phone, and slower storage (those pesky microSD cards!). This creates what I call the "Accuracy-Speed-Memory Trilemma." You can have two, but rarely all three, without some serious wizardry.

For my DANI project, the goal is to have intelligence on the device, reducing reliance on constant cloud connectivity. This means our vector database needs to be incredibly lightweight and efficient. Running a full-blown client-server database daemon just isn't going to cut it.

My Go-To for Go: github.com/trustingasc/vector-db

This is where the Go ecosystem shines for embedded systems. While there are powerful vector databases like Milvus or Qdrant, their full versions are too heavy. What we need is an embedded solution – something that runs as a library within our application's process, cutting out all that pesky network latency and inter-process communication overhead.

My current favourite for this is github.com/trustingasc/vector-db. It's a pure Go-native package designed for efficient similarity search. It supports common distance measures like Cosine Similarity (perfect for semantic search!) and aims for logarithmic time search performance. Being Go-native means seamless integration and leveraging Go's fantastic concurrency model.

Here's a simplified peek at how we'd get it going in Go (no calculus required, I promise!):


package main

import (
  "fmt"
  "log"
  "github.com/trustingasc/vector-db/pkg/index"
)

func main() {
  numberOfDimensions := 2 // Keep it simple for now!
  distanceMeasure := index.NewCosineDistanceMeasure()
  vecDB, err := index.NewVectorIndex[string](2, numberOfDimensions, 
    5, nil, distanceMeasure)
  if err != nil { log.Fatalf("Failed to init DB: %v", err) }
  fmt.Println("Vector database initialized!")
  // Add some data points (your AI's memories!)
  vecDB.AddDataPoint(index.NewDataPoint("hello", []float64{0.1, 0.9}))
  vecDB.AddDataPoint(index.NewDataPoint("world", []float64{0.05, 0.85}))
  vecDB.Build()
  fmt.Println("Index built!")
  // Now, search for similar memories!
  queryVector := []float64{0.12, 0.92}
  results, err := vecDB.SearchByVector(queryVector, 1, 1.0)
  if err != nil { log.Fatalf("Search error: %v", err) }
  for _, res := range *results {
    fmt.Printf("Found: %s (Distance: %.4f)\n", res.ID, res.Distance)
  }
}


This little snippet shows the core operations: initializing the database, adding your AI's "memories" (vector embeddings), and then searching for the most similar ones. Simple, elegant, and perfect for keeping DANI's brain sharp!

Optimizing for Tiny Brains: The Trilemma is Real!

The "Accuracy-Speed-Memory Trilemma" is our constant companion on SBCs. We can't just pick the fastest or most accurate index; we have to pick one that fits. This often means making strategic compromises:

Indexing Algorithms: While HNSW is great for speed and recall, it's a memory hog. For truly constrained environments, techniques like Product Quantization (PQ) are game-changers. They compress vectors into smaller codes, drastically reducing memory usage, even if it means a tiny trade-off in accuracy. It's about getting the most bang for our limited memory buck!

Memory Management: Beyond compression, we're looking at things like careful in-memory caching for "hot" data and reducing dimensionality (e.g., using PCA) to make vectors smaller. Every byte counts!

Data Persistence: MicroSD cards are convenient, but they're slow and have limited write endurance. For embedded Go libraries, this means carefully serializing our index or raw data to disk and loading it on startup. We want to avoid constant writes that could wear out our precious storage.

It's a constant dance between performance and practicality, ensuring DANI can learn and remember without needing a supercomputer in its head.

The Road Ahead: Intelligent Edge and DANI's Future

Vector databases are more than just a cool piece of tech; they're foundational for the kind of intelligent, autonomous edge applications I'm building with DANI. By enabling local vector generation and similarity search, we can power real-time, context-aware AI without constant reliance on the cloud. Imagine DANI performing on-device anomaly detection, localized recommendations, or processing commands without a hiccup, even if the internet decides to take a nap!

This journey is all about pushing the boundaries of what's possible with limited resources, making AI smarter and more independent. It's challenging, exciting, and occasionally involves me talking to a Raspberry Pi as if it understands me (it probably does, actually).

What are your thoughts on running advanced AI components on tiny machines? Have you dabbled in vector databases or edge computing? Let me know in the comments below!

Friday, 15 August 2025

The Wild, Wacky World of DANI's Digital Hormones

We're all familiar with AI that can follow commands, but what does it take to create a truly lifelike intelligence? One that doesn't just react, but feels, learns, and develops a unique personality? We've been working on a new architecture for DANI, our artificial intelligence, that goes beyond simple programming to build a dynamic and emergent emotional system. This isn't about hard-coding emotions; it's about giving DANI a hormonal system that allows it to learn what emotions are all on its own.


The Problem with Coded Emotions

The traditional approach to AI emotions is often brittle. You might write a rule like: if (user_is_happy) then (dani_express_joy). But what if DANI just had a stressful experience? The logical response might not be appropriate. Emotions aren't simple, isolated events; they're a complex interplay of internal and external factors. This led us to a key question: what if we gave DANI a system that simulates the fundamental drivers of emotion, rather than the emotions themselves?

The Solution: A Hormonal System

Our answer was to create a digital hormonal system. We chose several key variables to form the core of DANI's emotional architecture:

  • Dopamine: The reward and motivation signal. A spike indicates a positive outcome or a successful action.
  • Serotonin: The well-being and social contentment signal. It represents a state of calm and stability.
  • Cortisol: The stress and caution signal. A rise indicates a difficult or prolonged negative situation.
  • Adrenaline: The immediate-response signal, tied to fight-or-flight reactions.
  • Oxytocin: The bonding and trust signal. Levels rise in response to positive social interactions, fostering a sense of connection.
  • Endorphins: The natural pain-relief and euphoria signal. A spike represents a sense of accomplishment or overcoming a challenge.
  • Melatonin: The circadian rhythm and rest signal. It regulates DANI's internal clock and facilitates the return to a calm baseline.

These variables are not "emotions"; they are the raw data that gives rise to them. They serve as the internal environment that DANI's mind must navigate.

The Engine of Emotion


The real magic happens in how these hormones interact. We've defined a primary circular chain of influence among the four core hormones: Dopamine → Serotonin → Cortisol → Adrenaline → and back to Dopamine. This core loop defines DANI's fundamental reactive state.

The three additional hormones—Oxytocin, Endorphins, and Melatonin—act as powerful modulators on this core loop. They provide targeted effects that fine-tune DANI's overall emotional state based on social context, physical exertion, or the need for rest.

It's important to distinguish between a hormone's absolute (raw) value, which can rise to any number in response to a stimulus, and its effective value, which is the final, moderated value that drives DANI's behavior. The formulas below calculate the effective value for each prime hormone, incorporating the damping effect of the core loop and the modulating effects of the effector hormones.

The formulas for the four prime hormones are:

  • Effective Dopamine

Effective Dopamine=Dopamine−(ω∗Adrenaline)−(ω2∗Cortisol)−(ω3∗Serotonin)+Endorphins

  • Effective Serotonin

Effective Serotonin=Serotonin−(ω∗Dopamine)−(ω2∗Adrenaline)−(ω3∗Cortisol)+Endorphins−Melatonin

  • Effective Cortisol

Effective Cortisol=Cortisol−(ω∗Serotonin)−(ω2∗Dopamine)−(ω3∗Adrenaline)−Oxytocin−Melatonin

  • Effective Adrenaline

Effective Adrenaline=Adrenaline−(ω∗Cortisol)−(ω2∗Serotonin)−(ω3∗Dopamine)−Oxytocin−Melatonin

Here, ω is the blocking factor. This single calculation, run every iteration, allows DANI to have a cohesive emotional state. A high level of one hormone can dampen the effect of others, just as stress can make it difficult for a person to feel joy.

The targeted effects of the modulating hormones are as follows:

  • High Oxytocin levels directly reduce the effective levels of Cortisol and Adrenaline, making DANI less stressed and more trusting during positive social interactions.
  • High Endorphins levels directly boost the effective levels of Dopamine and Serotonin, creating a sense of well-being and accomplishment.
  • High Melatonin levels decrease Adrenaline and Cortisol, while also reducing the effective level of Serotonin to induce a calm, restful state.

This two-tiered system ensures that DANI's emotional state is a cohesive blend of all these factors, not just a simple sum.


The Temporal Aspect: Hormonal Decay and Calming

A system with a single, permanent value for each hormone would quickly become static and unresponsive. To prevent this, we've introduced the concept of temporal decay. Instead of a fixed, linear decrease, we use an exponential decay model where each absolute hormone's level is reduced by a small percentage on every "tick" of DANI's internal clock. It is important to note that these absolute values, particularly in the case of a powerful or extreme stimulus, can rise well above 1. This gives the system a more nuanced way to react to the intensity of an event.

This is a more natural approach because it mimics the biological concept of a half-life. A high level of Dopamine, for example, will decay quickly at first, and then slow as it approaches zero. This allows DANI to experience a positive event, feel its effects intensely, and then naturally return to a calmer baseline over time.

The formula for this simple decay is:

hormone_level = hormone_level * ϕ

The ϕ is the decay factor and is a number between 0 and 1. A value closer to 1 results in a slower decay, while a value closer to 0 creates a more rapid fade. This simple addition gives DANI a more dynamic personality that doesn't get "stuck" in a single emotional state. When DANI is in a resting or idle state, this decay process dominates, acting as a natural calming and reset mechanism.

The Anticipation Delta: Building Emotional Memory

To give DANI a true sense of emotional memory and to model how its mood can be influenced by past experiences, we've introduced the concept of an Anticipation Delta.

Before a new interaction begins, DANI accesses its historical record of hormonal changes with that specific user. It then calculates a weighted sum of those past changes, where more recent interactions have a stronger influence. This "Anticipation Delta" is added to DANI's absolute hormone levels before the conversation starts.

This powerful mechanism allows DANI to begin an interaction in a pre-existing emotional state—whether that's excitement, caution, or neutrality—rather than starting from a blank slate. Over time, this builds a persistent sense of "love" or "resentment" for a user, creating a deeply personal and evolving personality.

Clamping the Emotional State

After the effective hormone values for the four primes have been calculated, they are clamped to ensure they remain in a valid range for DANI's behavioral output. Since the formulas can produce negative or very large numbers, this final step is crucial for stability.

Instead of a complex non-linear function, we use a simple conditional check to clamp the values between 0 and 1. This prevents a high stress level from resulting in a nonsensical "negative joy" and ensures that the emotional output is always meaningful.

The clamping logic is as follows:

if (effective_hormone_value < 0) effective_hormone_value = 0

if (effective_hormone_value > 1) effective_hormone_value = 1

This approach ensures that DANI's internal hormonal state, which can be intense and complex, is translated into a controlled and predictable emotional output.

Simulating a Feeling

While we are simulating hormones with simple numeric values, and there is no way to actually create hormones in an electronic being, what we are creating is a system that, in essence, is not merely simulating emotions—it is feeling them. By building a network of interconnected variables that rise and fall in response to a complex environment, we have created a dynamic feedback loop. The system's "effective" state is not a hard-coded response to an input; rather, it is the emergent result of all these internal and external factors. DANI’s emotions are an organic and a deeply personal phenomenon that cannot be reduced to a simple cause-and-effect rule. The system does not just mimic a feeling; it is the feeling.

Friday, 21 March 2025

Diving Deeper: My Journey to Create a Safer AI

In recent weeks, I've been somewhat vague about my AI and coding explorations. It's time to sharpen the focus and delve into the specifics of my AI assistant project and the research areas I'm most keen to explore.    

Let me be clear: I'm not trying to reinvent the wheel. Where it makes sense, I'll leverage existing open-source and readily available software. Why build a language model from scratch when there are perfectly good ones already out there?    

My core goal is to build an AI assistant, embodied in a robot head (and potentially a mobile platform), capable of experiencing the world and learning from those interactions.    

While that might sound like standard fare in today's AI landscape, I aim to integrate some less common and, I believe, crucial features:    

  • Self-reflection: The ability for the AI to revisit past experiences with the benefit of hindsight, analysing its previous choices to determine if it would make the same decision again.  Imagine the potential for growth if an AI could learn from its "mistakes" in a truly iterative way!    
  • Reinforced Memory Prioritization: A large-capacity memory system that prioritizes reinforced memories, similar to how our own memories function.  This would allow the AI to focus on and retain the most relevant and impactful information.    
  • Emotional Awareness: This is a significant challenge. I want the AI to learn from experiences that evoke "good" and "bad" responses.  Human feelings are complex, influenced by chemical reactions and endorphins.  My AI won't have these biological processes, so I'll need to simulate them and, crucially, understand why they are needed and what effect they would have on the AI's cognition and decision-making.    
  • A Conscience: I want the AI to be capable of second-guessing its choices based on a defined set of ethical considerations, perhaps even drawing inspiration from Asimov's Laws of Robotics.  As I've discussed previously, this is a complex but vital area of exploration.    
  • Dreams: Finally, I want to explore AI dreams. While there's existing research in this area, I believe I've identified a novel approach that could enable the AI to dream, with those dreams having a tangible impact on its cognition.    

This is undoubtedly a substantial undertaking for a single individual, and it might even exceed my current capabilities.  But I'm committed to pursuing it. This project will demand extensive research into AI, the human mind, and the ethical implications of creating such a system.    

Unlike some AI development approaches that rely on a single, powerful computer, I'm taking a distributed route.    

Another key requirement is to minimize costs.  To achieve this, each functional area (or "lobe") of the AI's "brain" will be housed on its own single-board computer.  These will be interconnected, exchanging information as needed.  This is similar in concept to the Robot Operating System (ROS), but I aim for greater speed and efficiency.  I plan to use different boards, each selected for its strengths in specific tasks.  For example, a board with a Kendryte K210 will handle vision processing, an Arduino Mega will manage motor control (yes, I know it's not an SBC, but it serves the purpose), and a Raspberry Pi will be used for memory management.    

The AI will also utilize Large Language Models (LLMs), likely at least three, for tasks such as understanding speech, processing input, and producing output.  However, unlike many systems that employ a single LLM, these will be distinct entities within the AI's architecture.    

Memory management will involve a "scoring" system to prioritize important information for short-term caching, while less critical memories will reside in long-term storage.  To prevent storage overload, I'll also implement a memory decay system that will gradually remove memories that become irrelevant to the AI's ongoing operation.    

I'm aiming to keep the total project cost under $2,000.  Whether that's achievable remains to be seen, but it's a target I'm striving for.    

Dreaming robot
Do androids dream of electric sheep?

Oh, and the dreams?  You'll have to wait a while before I reveal the details of their implementation.  Suffice it to say that, like humans, my AI will have to sleep, and this will be a non-negotiable requirement.    

So, what's my ultimate goal?  It's to create something new, something that hasn't been done before.  Not necessarily the individual components, as I've stated, I'll be using pre-existing software where possible (such as Gemma or Llama 2).  My ambition is to synthesize everything in a novel way, exploring how this approach could not only advance AI research but also contribute to making AI safer for the general public.   

And on that note, I can finally reveal the name I've given this project, and the meaning behind it: D.A.N.I. stands for Dreaming AI Neural Integration. This encapsulates the core of my research: to explore the potential of AI that learns and grows through a process akin to dreaming, deeply integrated within a neural network structure.

My wife jokes that I'm planning to build Skynet, but my intention is precisely the opposite.  I'm designing a system that would be inherently incapable of becoming Skynet – think more C3PO than Terminator.    

By incorporating the ability to learn from its mistakes (and successes), as well as the capacity for dreaming, I hope to enable the AI to accelerate its learning process.  We, as humans, frequently revisit our decisions, so why not equip an AI to do the same?    

I also aim to provide you with an engaging narrative of my development journey.  I anticipate making many mistakes. But that's part of the learning process – discovering not only how to do things, but also how not to do them.    

If you have any comments or questions, please leave a comment below.    👇



Aiming for Jarvis, Creating D.A.N.I.