Every CRM vendor claims to have AI-powered features. While 65% of businesses have adopted AI-powered CRMs, only 17% of them actually use more than two AI features. The reason isn’t just reluctance to adopt new technologies. There is fundamental confusion about what AI actually is when it comes to CRM.
Marketing language is often unclear when it comes to which of their CRM features use AI, and some CRM vendors even conflate basic automation with AI.
For technical buyers, this makes evaluating CRM tools difficult. It’s especially confusing for less technical buyers. How do you evaluate CRM AI capabilities when vendors aren’t clear about the types of AI they’re using and which features they use it for?
This guide provides evaluation criteria for three types of CRM capabilities marketed as AI. Plus, you’ll get questions to ask vendors, proof-of-concept tests, and a scoring rubric for technical evaluation.
Three types of “AI” in CRM (and why vendors conflate them)
Before assessing the AI functions of a CRM, you must determine the AI types offered by the vendor.
Type 1: Rule-based automation (not true AI)
Rule-based automation covers things such as if/then logic, the automation of workflows and auto-populated fields. It is deterministic in that the same output is created for the same input. This type of automation does not require learning or training, however, does not benefit from improvement over time.
One important caveat: Rule-based automation isn’t technically AI. It’s a capability that’s been around for decades, but today it’s often confused with true AI offerings. While it has real value on its own, it’s not true AI.
Here’s an example of rule-based automation at work: “When a lead fills out a form, create a lead and send an email sequence.”
Type 2: Generative AI
Generative AI is the most common type of true AI in CRM, and what most people think of when they think of AI. In a CRM system, this technology oversees features such as summarization, content generation, research aid, and conversational interfaces.
This kind of AI employs pre-trained models like ChatGPT and Claude. It is instantaneous and gets better the more detailed the prompts and instructions.
Email drafting, meeting summarization, pipeline design, and lead research are a few examples of the generative AI capabilities found in a CRM system.
Type 3: Predictive machine learning
Predictive machine learning is a more advanced type of AI capability that requires training custom models on your historical data. They take several months to become accurate and improve over time as your CRM provides them with new data.
CRM features that use this type of AI may include lead scoring, forecasting, churn prediction, deal probability estimation, and next-best-action recommendations.
For example, a predictive machine learning feature could tell you the conversion probability of a lead based on patterns from the history of past deals in your CRM.
Why vendors conflate them
All three of these types are often marketed simply as “AI-powered CRM” without clearly defining which features use which types of AI. This creates confusion and makes technical evaluation difficult. Whether intentional or not, this can benefit vendors as buyers may believe they’re getting more AI benefits than they truly are.
While over 70% of enterprise organizations have deployed AI-powered CRMs, about 45% of CRM data isn’t AI-ready, meaning it lacks the quality and completeness to support predictive machine learning. The reality of AI implementation often doesn’t match the benefits promised.
When evaluating CRM vendors, look for ones that clearly identify which CRM features use AI and how they work. Nutshell, for example, explicitly lists its AI features, including generative AI capabilities like timeline summarization, meeting transcription, email drafting, and lead research.
When vendors provide clear information about their CRM features, this simplifies evaluation and sets realistic expectations.
Technical evaluation framework by AI type
When evaluating AI CRM capabilities, adjust your approach based on the type of AI you’re assessing.
Evaluating rule-based automation
If you’re evaluating rule-based automation, evaluate it as you would any workflow automation tool. These are a few important considerations:
Capabilities: What actions can you automate? What can you set as triggers?
Logic Transparency: Are you able to view and edit rules of the automation?
Flexibility: Adjusting thresholds or creating conditional logic—would you be able to do that?
Error Handling: What does the system do when faced with issues like edge cases or a lack of data?
Maintenance Burden: What is the maintenance like for the system? Will it require a lot of maintenance? Will you be able to change rules as your needs change?
Since this type of feature isn’t real AI, you don’t have to consider the learning, improvement, or adaptation capabilities.
Evaluating generative AI
If you’re assessing generative AI, your approach will differ. You’ll want to assess elements like the following:
- Functionality: Which language models do the features use? This will give you an idea of functionality and what the output might be like.
- Data Privacy: When it comes to data privacy, you need to make sure whether CRM data will be shared to third parties. If so, how will those parties use that data? Will those parties use the CRM data to train their models? Only work with CRM vendors who have short, easy to understand data privacy policies.
- Customization: How customizable are the features? Can you easily adjust prompts, output parameters, and tool behaviors?
- Output quality and consistency: Output quality, of course, is essential. Test the tools with multiple real examples to make sure it consistently produces quality outputs.
Another vital consideration is the pricing model used. Many vendors meter every action with a credit system, which can make predicting costs challenging. Team members may also hesitate to use the features because they worry about hitting limits.
Some CRM vendors use more straightforward pricing models. The AI CRM Nutshell, for example, offers unlimited AI assists with predictable monthly outcome pools, which simplifies pricing.
Evaluating predictive machine learning
Predictive machine learning features require the most rigorous technical evaluation of the three types. Here are some of the aspects you’ll want to consider:
- Training data transparency: What size dataset does the model need to produce accurate results? Is this information clearly presented by the CRM vendor?
- Explainability: In terms of explainability, do they clarify the basis of the system’s predictions? Do they offer feature-level attribution, confidence intervals, audit trails to meet regulatory requirements, and a documented methodology?
- Adaptation mechanisms: What type of adjustment mechanisms are in place? Describe how the system works to fine-tune and better its results over time. Explain the retraining cycles and the drift detection and correction mechanism, if present.
- Independent verification capability: In terms of independent verification, can you export data to test the predictions made by the system?
Another factor to consider with this type of AI feature is the quality of the data you have available. According to data from Gartner, 60% of AI projects will be abandoned in 2026 because they’re not supported by AI-ready data. To get the most out of this type of AI CRM feature, make sure you have 12-plus months of clean data before investing.
Vendor questions that reveal AI type
If you’re unsure what type of AI a CRM is using, asking the CRM vendor these questions can help you figure it out.
Can you show me the training data requirements?
If the feature is using rule-based automation, no training will be needed. You simply configure the desired rules.
Generative AI employs models that are pre-trained and hence can be used directly on your data.
Predictive ML usually needs about 200 to 500 task-specific historical records to perform the intended prediction.
How long until this feature becomes accurate or useful?
Rule-based automation and generative AI features should be accurate immediately since they’re using configured rules and pre-trained models. Predictive machine learning may take six to 18 months to learn from your data.
What happens when I expand to new market segments or product lines?
With rule-based automation features, you may need to update rules to include new information, but otherwise, everything should continue to work as normal.
Features that use generative AI should continue working with no changes.
For predictive features, accuracy will decrease until the model retrain using the new information.
Can you explain why the system produced this specific result?
When it comes to rule-based automation, looking at the rules should clearly explain the outcome.
For generative AI, you’ll be able to see the prompt and likely the reasoning process, but it’s less clear why the model produced a specific result.
With predictive AI features, vendors should have the ability to provide feature attribution and confidence scores, although this requires some technical knowledge.
Is this feature using pre-trained models or learning from our data?
Rule-based automation does neither, since it’s simply deterministic logic. Generative AI uses pre-trained models, while predictive features learn from your historical data.
Scoring rubric for comparative evaluation
Use this framework for objective vendor comparison across AI types.
| Evaluation criterion | Weight | Scoring guidelines |
| AI type transparency | 20% | 1-3: Vague “AI-powered” claims4-6: Mentions capabilities but unclear type7-8: Clear type identification9-10: Detailed technical documentation by type |
| Match to data readiness | 25% | 1-3: Requires data you don’t have4-6: Partial fit7-8: Matches current state9-10: Works now AND scales with growth |
| Output quality and accuracy | 25% | 1-3: Unusable outputs4-6: Requires heavy editing or validation7-8: Usable with minor tweaks9-10: Production-ready |
| Data privacy and security | 15% | 1-3: Unclear data handling4-6: Basic disclosures7-8: Clear privacy controls9-10: Full audit trail plus compliance certifications |
| Pricing transparency | 15% | 1-3: Hidden costs or complex metering4-6: Documented but complex7-8: Clear, predictable pricing9-10: Transparent with no surprises |
To use this rubric, first figure out what type of AI you’re evaluating. Then, score the CRM using the rubric and criteria that are appropriate for the CRM type. For example, don’t assess generative AI on its predictive capabilities. Also, feel free to adjust weights based on your priorities.
The bottom line for technical buyers
AI CRM is a phrase that’s thrown around a lot, but it can mean many different things. There are three main types of AI in CRM you’re likely to encounter — rule-based automation (which isn’t true AI), generative AI, and predictive machine learning.
All three types are valuable, but evaluating them requires different approaches.
The first thing you should evaluate is vendor transparency about the kind of AI they use. A lack of transparency here signals either a lack of honest communication or technical depth.
Data scientists understand how to conduct technical evaluations, and they can use these skills when assessing AI features in CRMs. When speaking with CRM vendors, request additional details about the system’s architecture, what the training entails, their validation approaches, and how much of their system is explainable.
When selecting an AI-driven CRM, keep in mind that a system being advanced and/or highly sophisticated does not make it the best option. In addition to assessing a system’s technological capabilities, assess a CRM’s alignment with your data readiness, its ease of fit with current workflows, and the likelihood of the system generating significant ROI within the timeframes you have established.
This strategy will enable you to set reasonable expectations for the AI capabilities of the CRM you select and will help you optimize your use of the system.
Looking for a leading AI-powered CRM to streamline your workflows? Give Nutshell a try. Sign up for a free 14-day trial—no credit card required.