AI in Practice: Test Early And Often

There is growing speculation that Salesforce might be cooling on AI. The change in attitude was reinforced at Davos last week, when Marc Benioff used his public appearances to share some fairly stern criticisms concerning the impact of AI chatbots on social media. So far, those views have not been reflected in Salesforce’s overall corporate strategy. Agentforce is still pitched as the future of the platform. However, there has been a subtle change of emphasis. AI is now seen as one pillar of a broader framework. This shift reflects the experiences of Salesforce customers working with the technology.

Test Multiple Times

Driving the updated narrative is one key problem. Designing an AI workflow takes longer than initially assumed. Integrating AI decision steps into an existing process is the easy part. Actually testing the AI is much more complicated. It’s a far more involved process than a traditional development project, requiring many more test runs. Generative AI will give a different output for every test, even if the same prompt is used. As such, it is not possible to test a scenario once and then mark it as pass or fail. Every test scenario needs to be tested multiple times, and each response graded on a sliding scale. That can be challenging given that assessing the quality of AI generated content is inherently subjective.

Before launching an AI project, it is extremely important to understand what an MVP looks like. Much like any content creation workflow, clear guidelines around the language, tone and content of AI output are needed. It doesn’t matter whether the AI is generating content for external use, internal sharing or even just for the next stage of a workflow. It is essential to decide what an acceptable result looks like for AI generated content. Once an ideal outcome has been agreed, be prepared for a phased roll out over an extended period. Too many AI projects fail because users don't have a clear vision for the type of content they want the AI to generate.

Be Focused

It’s hard to judge whether a pilot is production ready. Create a checklist of everything the AI needs to consider before starting to test it. That checklist needs to be prioritised, with the must have items clearly marked. Make sure everyone is realistic about what production ready actually looks like, and what the AI needs to consider in order to deliver business value. There needs to be an agreed level of error or acceptable hallucination. Otherwise, it’s very easy to get sidetracked in the pursuit of perfection and ultimately never launch.

Keeping AI focused on a single task is essential. The longer the prompt, the more likely it is to get confused. AI doesn’t understand the overarching operation environment or underlying user intent. It only knows the instructions and training data that have been shared with it. Splitting an AI into multiple agents significantly increases accuracy, while also making testing much easier. A multi-agent approach allows the output of each step to be evaluated independently, and the instructions for each step tweaked independently.

Include A Feedback Loop

Much like a software project, AI is a never the finished article. You always need a mechanism for monitoring AI outputs and training the model, even after go live. AI is not a set and forget process. New errors will crop up during day-to-day use. Salesforce have learned the importance of feedback loops and prompt refinement. It’s a critical part of their own AI development toolkit. AI is an iterative process. Testing an AI workflow takes vastly more effort than testing a rule based workflow. That QA process carries on after launch.

In every AI workflow, there needs to be a mechanism for teaching the AI new information. Ideally, that would be automated using machine learning. An ideal agent would include a feedback loop showing the AI the downstream impact of its output. However, that is often difficult to set up in practice. Even so, that should be done wherever possible. It saves a lot of manual time and effort. Without an automated learning process, there are no efficiency savings from AI. If feedback loops cannot be built-in, then human-in-the-loop processes are needed. This is about more that just quality assurance. Regular updates to training data are needed too. Those updates should be accompanied by another round of testing.

Consider The Limitations

While AI is still seen as central to the future of the technology industry, there has been a subtle change of emphasis. Rule based workflows have become a key part of every AI solution, including platforms such as Agentforce that were supposed to replace them. This is something that many people have wanted to happen. AI can’t be trusted to follow fixed instructions in every execution, which is why the rule based workflow is needed to actually deliver any actions suggested by AI. Agentic AI isn't going away. It's just that developers are evolving their AI features to fit the limitations of current AI technology. Ops team working with the technology need to follow their lead.

Salesforce Expand Marketing Cloud

Salesforce have a new Marketing Cloud. Originally pitched as an AI-driven agentic marketing solution, Marketing Cloud Next also replaces both of Salesforce's existing marketing products. That means Pardot can take advantage of the SMS, WhatsApp and non-marketing email capabilities available in Marketing Cloud. These new Pardot channels are among the new features in the Spring '26 Salesforce release, along with several new integrations that make it easier to use Pardot in parallel with Marketing Cloud Next.

Operationalising ABM in the Real World: Your Step-by-Step Guide

Finding it difficult to navigate the complexities of ABM. A new eGuide from CRMT Digital cuts through ABM hype, explaining how to implement and run an ABM program in practical terms. Boost ROI with practical approaches to account-based targeting, personalisation and engagement.

Marketing Operations Roadmap Matrix

A Marketing Operations Roadmap is an essential strategic tool that aligns marketing priorities with your overall business goals. Any roadmap begins with understanding where your marketing organisation is right now, and then defining where you want to get to. This free Marketing Operations Roadmap Matrix from CRMT Digital allows you to do exactly that.