Agent Memory on Azure: Redis or Cosmos DB?

An agent with no memory starts every conversation from zero. Give it memory and you have a new question: where does that data live?

Two Azure services come up first, Azure Managed Redis and Azure Cosmos DB. Microsoft documents both for agent workloads, and the pages say different things. This post lays out what each page actually covers, gives a rule of thumb for choosing, and lists the mistakes Microsoft's own documentation warns about.

Three jobs, not one

"Memory" covers three different things, and they have different needs:

1. Conversation history. The ordered messages of a session, so the agent can continue after a restart.
2. Memory and state. Facts retrieved into the agent's context, and the state of a long-running workflow.
3. A semantic cache. Reusing an earlier answer when a new prompt means roughly the same thing, to save tokens and time.

Microsoft's Agent Framework memory guide (updated 9 October 2026) shows the pieces. By default an agent keeps history in an in-memory provider or in the underlying AI service. For persistence you plug in a history provider or a context provider, and the guide shows combining several. It warns that only one history provider should load messages, so you don't replay several stores into the same request.

What Microsoft documents for each

Job	Redis	Cosmos DB
Conversation history (Agent Framework)	RedisHistoryProvider in the Python package	CosmosDBHistoryProvider (Python) and ChatHistoryProvider (.NET), per the Cosmos DB AI integrations page
Retrieving memories into context	RedisContextProvider	Not listed for Agent Framework on that page. LangGraph has a long-term memory store, optionally with vector search
Workflow checkpoints	Not in the package's component table	CosmosDBWorkflowCheckpointStorage (Python) and CheckpointStore (.NET)
Semantic cache	A tutorial for Azure Managed Redis	A concept page with sample apps and labs

Two cautions about that table. The Redis package is pre-release: its README installs it with pip install agent-framework-redis --pre, and marks two of its components, RedisCollection and RedisStore, as experimental. The repo also ships samples for basics, conversations and sessions, including one against Azure Redis.

And the integrations page, which covers Azure Cosmos DB for NoSQL, describes the service as one persistence layer for vector search, chat history, semantic caching, agent state and long-term memory. That is the service's range, not a promise that every framework has every connector. The table shows what the page lists for Agent Framework specifically.

A rule of thumb

This part is our judgment. Microsoft doesn't prescribe a split, and each service can do more than the table shows.

• Redis for data you can afford to lose and rebuild. A semantic cache is the clearest case: a cache miss costs you a model call, not your data. Short-term memory fits too.
• Cosmos DB for data you need to keep and query. Conversation history you may need to audit, agent state, and workflow checkpoints that let a long job resume.

The two can sit in one solution. If you can only run one, pick by which job hurts more to get wrong: losing a cache is cheap, and losing history or state isn't.

Before you build your own memory store, note that the same Agent Framework page also shows a Foundry memory provider, linked to "Microsoft Foundry managed semantic memory." We have not tested it, but it belongs on the list of options.

The third option: put the cache in the gateway

Microsoft's Redis tutorial says semantic caching is "an application or gateway pattern, not a separate Azure Managed Redis managed feature." Azure Managed Redis supplies the Redis endpoint and vector search. You choose where the cache logic runs:

• In your application, using the RedisVL library, when you want control over thresholds, partitions and metadata.
• In Azure API Management, using its llm-semantic-cache-lookup and llm-semantic-cache-store policies, with Azure Managed Redis as the external cache, when LLM traffic already flows through the gateway and you want one policy for every team.

The tutorial's prerequisites matter for planning. The RediSearch module has to be enabled when the Azure Managed Redis instance is created, and you can't add it to an existing one. RediSearch also needs the Enterprise clustering policy and the NoEviction eviction policy.

Mistakes to design out

All of these come from Microsoft's own pages.

• Caching without conversation context. Microsoft's Cosmos DB page gives an example. One user asks for the largest lake in North America, then "What is the second largest?" Another user asks about the largest stadium, then asks the same follow-up and gets the lake answer from the cache. A cache key has to include the conversation, not just the last prompt.
• One cache for everyone. The Redis tutorial says not to share an unpartitioned cache across callers. Partition by tenant, user group or subscription, and include the model, embedding model and prompt version, so a change to any of them doesn't serve stale answers.
• Copying thresholds between systems. RedisVL uses a cosine distance from 0 to 2, and API Management's score-threshold runs from 0.0 to 1.0. Validate the units and behavior before you reuse a number.
• Caching what shouldn't be cached. Microsoft's guidance is to avoid prompts or responses that are highly personalized, regulated, safety-critical or time-sensitive, or that depend on tool or transaction side effects.
• Skipping the test set. Microsoft's validation steps start with prompts that should match and prompts that must not, then review false positives and stale answers.

Frequently Asked Questions

Should I use Redis or Cosmos DB for AI agent memory on Azure?

Microsoft documents both, and they can be combined. As a rule of thumb, which is ours and not Microsoft guidance, use Redis for fast, replaceable data such as a semantic cache and short-term memory, and Cosmos DB for data you need to keep and query, such as conversation history, agent state and workflow checkpoints.

Is semantic caching a built-in Azure Managed Redis feature?

No. Microsoft's tutorial calls semantic caching an application or gateway pattern, not a separate managed feature. Azure Managed Redis provides the Redis endpoint and vector search, and the RediSearch module must be enabled when the instance is created.

What should you not put in a semantic cache?

Microsoft's guidance is to avoid caching prompts or responses that are highly personalized, regulated, safety-critical or time-sensitive, or that depend on side effects from tools or transactions, because a cached answer can be incorrect, outdated or unsafe for the current request.

Sources

1. Step 4: Memory & Persistence (Agent Framework): Microsoft Learn, updated 9 October 2026
2. Agent Framework Redis package and Redis context provider samples: microsoft/agent-framework on GitHub, MIT licence, not archived; the package installs as a pre-release
3. Tutorial: Use Azure Managed Redis as a semantic cache: Microsoft Learn, updated 16 September 2026
4. Semantic Cache for Large Language Models (Azure Cosmos DB): Microsoft Learn, updated 27 April 2026
5. Integrations for AI (Azure Cosmos DB): Microsoft Learn, updated 18 August 2026

This is general information, not legal, security or compliance advice. Preview and pre-release status and package contents change, so check the sources before you commit to a design.

Where to go next

• Not sure where your platform stands? Take the free Agentic AI Readiness Scorecard. It takes a few minutes and needs no email.
• Want the ownership and cost questions in a list? Use the free Agent Platform Production Checklist.
• Choosing a framework first? See Microsoft Agent Frameworks Compared.
• Want a second pair of eyes on your agent platform? Bring your score to a Discovery Call and we will scope the Agent Platform Readiness Review with you.