scripod.com
Building Semantic Memory for AI With Cognee

Overview

Shownote

Highlights

Transcript

Chapters

Pins

Building Semantic Memory for AI With Cognee

AI Engineering Podcast

2024/11/25
AI Engineering Podcast

AI Engineering Podcast

2024/11/25
OverviewShownoteHighlightsTranscriptChaptersPins

Shownote

Summary In this episode of the AI Engineering Podcast, Vasilije Markovich talks about enhancing Large Language Models (LLMs) with memory to improve their accuracy. He discusses the concept of memory in LLMs, which involves managing context windows to enhan...

Highlights

In this episode of the AI Engineering Podcast, host Tobias Macey speaks with Vasilije Markovic about the evolving challenge of memory integration in large language models (LLMs). As LLMs become more central to complex applications, maintaining context and long-term knowledge remains a persistent hurdle. Markovic shares insights from his experience building Cognee, an open-source semantic memory engine designed to enhance how AI systems retain and retrieve information over time.
00:19
Vasilije built a memory engine for AI apps in the past year.
01:39
Memory in LLM systems is based on in-context learning, enabling efficient operation at low cost.
03:05
Catastrophic Forgetting occurs when LLMs fail to retain prior knowledge after updating prompts.
05:06
Hierarchical memory improves retrieval efficiency in RAG stacks.
06:52
Graph RAG offers a more structured approach to knowledge retrieval
10:10
Semantic memory layer based on human cognition boosts LLM performance.
14:52
Cognee can be used as short-term memory in the stack, reducing LLM calls.
17:15
The speaker deployed an initial product on AWS using the Atkinson-Schiffrin model, Neo4j, and VB8.
23:32
Anticipate increased use of LLMs as costs decrease
29:53
Cognee supports deployment on AWS and Kubernetes with minimal system disruption.
34:34
Cognee secures $1.5 million funding for sustainable solutions in complex domains
42:06
Manual work may be automated with proper technology solutions like RAG and GraphRAG.
51:39
Relational tools may persist, vector stores could be replaced by open-source alternatives.

Chapters

Introduction to AI Engineering Podcast
00:00
Interview with Vasily Markovich
00:19
Understanding Memory in LLM Systems
01:39
Challenges of Forgetting in LLMs
03:05
Multi-Turn Interactions and Context Management
05:06
Hierarchical Memory in LLM Applications
06:52
Semantic Memory and Cognitive Science
10:10
Architectural Components for Semantic Memory
14:51
Development and Evolution of Cogni
17:15
Data Structures and Ontologies in LLMs
23:32
Integrating Cogni into System Design
29:29
Personalization and Use Cases for Cogni
34:34
Navigating Unknowns in AI Ecosystem
38:40
Potential Applications of Cogni
42:06
Lessons Learned in Building Cogni
46:36
Future Plans for Cogni
48:19

Transcript

Tobias Macey: Hello, and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems. Your host is Tobias Macey, and today I'm interviewing Vasilije Markovic about adding memory to LLMs to imp...