Notion 3.0 / Notion AI Follow Up
Not too long ago, I wrote about Notion’s announcement of Notion 3.0 (even though the current app version is 4.x… But whatever)- https://barnes.tech/blog/notion-30/ ↗. I was optimistic about what they announced. In a nutshell, they added “agents” and significantly updated the AI that powers Notion AI. Notion AI can work with Notion pages or databases but can also connect to all of the other data connectors that you have connected to Notion. Sounds great, and the demos looked good. After testing it out for a day or so I was fairly impressed.
Since then I’ve been using it a fair amount more, and it has been a pretty mixed bag for me. Sometimes it seems to do exactly what they demoed. I can reference a bunch of data, briefly explain what we are planning to do for a project, give it a good example of a well designed project proposal, and it can do an excellent job filling out at least a rough draft of a project proposal based on all that data. It has legitimately saved me several hours of busy work already.
But sometimes it completely shits the bed. It’s very weird. I know AI hallucination pretty well, but as long as you know how to craft prompts, give examples, and are really explicit with what you’re looking for, Claude and OpenAI are pretty good at sticking to the parameters you give it. Notion seems to occasionally just completely lose it’s mind. I’ve now twice had it completely ignore instructions I’ve given it, specifically about content on a page I want left as-is, or formatting I want to remain the same, and completely messes up a page. I’ve now twice had it delete embedded files that I specifically told it in the prompt to leave exactly as-is. Honestly, using a Notion MCP server and just having Claude Code or OpenAI Codex interact with Notion has been much more successful so far, which is strange to me since Notion is using both models under “Notion AI”.
The idea they have here I think is solid, and Notion seems like a great place to use AI when a lot of time I’m just trying to consolidate data, summarize something, or do a bunch of busy repetitive work, but in it’s current state, you better be careful. It feels like this needs to bake in the oven for a while longer before they push it too far.