531: Using AI in risk-adverse industries – with Matt Coatney


Episode Artwork
1.0x
0% played 00:00 00:00
Mar 17 2025 46 mins   21

AI in product management – perspectives from the legal industry, education, and entrepreneurship



Watch on YouTube





TLDR



In my recent conversation with product executive and former colleague Matt Coatney, we explored how artificial intelligence is transforming product management and innovation. The technology has evolved dramatically in the past decade, from fragile, expensive systems to powerful tools that integrate seamlessly into workflows. Product managers can leverage AI for everything from customer research and brainstorming to prototyping and workflow automation. While organizations must balance specialized versus general AI tools and address concerns like hallucinations and data privacy, the benefits for productivity and innovation are substantial. The most successful implementations focus on solving real customer problems and seamlessly integrate into existing workflows.



Key Topics




  • Using AI as a brainstorming partner to overcome creativity blocks


  • Accelerating product development with AI-powered prototyping tools


  • Integrating AI into product management platforms and workflows


  • Balancing specialized AI products versus general-purpose models


  • Managing AI hallucinations and verification challenges


  • Learning from AI adoption in risk-averse industries like legal


  • Impact on mentorship and professional development


  • Future trends: local AI models and data privacy



Introduction



In this episode, we had a free-form discussion. My guest doesn’t know what I’m going to ask him and I don’t know what he is going to ask me. Our goal is to make the discussion valuable for product managers, leaders, and innovators.



Joining me is a former colleague, Matt Coatney. We worked together on an important product for LexisNexis. I went on to teach graduate courses in innovation and coach product managers and leaders in organizations, while Matt got more involved in Information Technology, leading professional services and consulting operations for a few organizations as well as serving as CIO for one of the large law firms in the US. His career started in AI systems some 25 years ago and today he continues learning about and applying AI and is also is a product executive.



The Current State of AI in Product Management



Matt asked about my observations of the effects of AI, from the perspective of a product manager, entrepreneur, and educator.



Last year at the Product Development and Management Association (PDMA) conference, three separate sessions featured AI tools specifically designed for customer research. This wasn’t just theoretical discussion. These were practical applications already being implemented by forward-thinking product teams.



At the PDMA conference, I participated in a workshop led by Mike Hyzy, where we completed what would normally be a 3-5 day Design Sprint in just three hours. Our team consisted of four humans and one AI companion, which functioned as a fifth team member. The AI was operated by someone skilled in prompt writing who understood the product space.



What impressed me most was how the AI accelerated our work. When we brainstormed customer problems, the AI helped us explore details we hadn’t considered. It suggested unmet needs, offered additional perspectives, and helped us develop a comprehensive view in a fraction of the time it would have taken traditionally. By the end of those three hours, we had developed a solid marketing description for solving a customer problem and created a decent prototype—impressive results for such a compressed timeframe.



Practical AI Applications for Product Professionals



Product managers can use AI tools as brainstorming and coding partners


On a personal level, I’ve found AI tools like Claude to be valuable as brainstorming partners. Rather than trying to craft lengthy prompts with all my requirements upfront, I’ve shifted to a more conversational approach. Kicking ideas around with AI helps me overcome the inertia of starting a task.



Matt shared similar experiences, noting that this brainstorming use case is often underappreciated. While many focus on productivity enhancements like email responses and content generation, the creative thinking support is particularly valuable for entrepreneurs and product innovators. Many professionals today lack the “water cooler culture” opportunities to casually discuss ideas with colleagues, especially with remote work becoming more common. AI tools help fill this gap, providing an always-available thinking partner.



We also discussed the prototyping capabilities of AI. Matt mentioned tools like GitHub Copilot for assisting developers, and V0, which can build functional web applications directly from human prompts. These tools allow people with little or no coding knowledge to write code.



For entrepreneurs and product managers, these prototyping tools address a common challenge: developing clear user workflows before engaging software developers. Taking time to create detailed prototypes helps clarify thinking and identify assumptions that might confuse users. AI accelerates this process, allowing teams to get clarity on user experiences sooner and save significant time and money during development.























AI Application







Value for Product Professionals






Brainstorming partner







Overcomes creative blocks and inertia






Idea validation







Tests concepts quickly without scheduling meetings






Prototyping assistance







Accelerates creation of user interfaces and workflows






No-code development







Allows faster proof-of-concept creation






Design iteration







Enables rapid exploration of alternative approaches






AI-Enhanced Product Management Platforms



The integration of AI into existing product management tools represents a significant opportunity for enhancing team effectiveness. ProdPad, one of the more popular platforms for managing product management work, and many competitors, have recently added AI capabilities—currently called Co-pilot—to their toolkit.



What makes these integrated platforms valuable is their ability to serve as a central repository for product information. They help teams maintain alignment with overall strategy, track progress toward objectives, and understand user stories. With AI enhancement, these platforms can now help identify gaps in strategy alignment, surface unmet customer needs based on existing data, and answer questions from stakeholders outside the immediate product team.



Matt and I discussed an important consideration when choosing AI solutions: whether to invest in specialized AI products or use general-purpose AI with custom prompts. Many specialized tools are essentially using the same large language models as ChatGPT but with carefully engineered prompts and workflows tailored to specific use cases.



For organizations making these decisions, Matt shared insights from his experience co-leading AI initiatives at his law firm. They’ve taken a tiered approach, using one solid general-purpose language model for most applications while investing in legal-specific AI products for their revenue-generating lawyers. For highly specific tactical use cases, they evaluate additional specialized tools—but only when the return on investment justifies the significant cost.




















AI Tool Approach







Pros







Cons






General-purpose AI (e.g., ChatGPT, Claude)







Lower cost, versatility, continuous updates







May require custom prompt engineering, less specialized






Industry-specific AI solutions







Optimized for domain-specific knowledge and workflows







Higher cost, potential duplication of capabilities






Integrated platform with AI features







Seamless workflow integration, centralized data







May have less advanced AI capabilities than specialized tools






Custom-built internal AI tools







Precisely tailored to organization’s needs







Resource-intensive to develop and maintain






Matt emphasized that the bar should be high for investing in specialized products when general tools can accomplish 90% of the required tasks. Organizations must consider whether the additional 10% improvement justifies spending five to six figures on multiple specialized tools, which could quickly add up to a million-dollar investment.



Workflow integration is important for successful AI implementation. Matt provided an example: if an organization has employees manually uploading invoices to ChatGPT, extracting data, and re-entering it into systems, they’re missing efficiency opportunities. The real value comes from automating these workflows to minimize manual steps.























Implementation Focus







Key Considerations






Model Selection







Balance between general and specialized capabilities






Integration Level







How seamlessly AI fits into existing workflows






Data Strategy







What information AI can access to maximize value






ROI Analysis







Justification for specialized AI investments






User Adoption







Support for different user groups based on needs






Matt also reflected on the current state of AI capabilities. He noted that today’s models are becoming so robust that it’s increasingly difficult to find use cases they can’t handle, at least for English language applications. This growing general applicability raises the bar for specialized solutions to prove their value.



We briefly touched on the debate around artificial general intelligence (AGI), acknowledging that while the term itself may be somewhat ambiguous, the general applicability of today’s AI tools is already impressive. This evolution has significant implications for how organizations approach their AI strategy, suggesting that for many use cases, the focus should be on integration and workflow rather than pursuing incrementally more powerful specialized models.



Addressing AI Challenges and Concerns



Implementing AI in product development isn’t without challenges. Matt and I explored several concerns that organizations must address to effectively leverage these tools.



We discussed the potential impact on professional development, particularly in apprenticeship-model professions. If senior staff rely on AI instead of junior team members for certain tasks, how will those juniors develop expertise? Matt raised this concern for lawyers and software developers, and we discussed its relevance for product management as well.



However, we identified a potential upside: AI could handle routine tasks that would previously occupy junior employees’ time, freeing them to engage in higher-value learning experiences. By eliminating basic tasks like fixing simple coding errors or catching obvious document mistakes, AI might actually create more meaningful mentorship opportunities focused on strategic thinking and core professional skills.



Product managers should be aware of risks of AI hallucinations
Product managers should be aware of risks of AI hallucinations


AI hallucinations—where models generate plausible but incorrect information—remain a persistent challenge despite recent improvements. I shared a personal experience using AI to analyze a detailed lease agreement. While the AI successfully identified unfavorable clauses and accurately referenced their location in the document, I was initially concerned it might fabricate issues. In other contexts, I’ve frequently encountered hallucinations where AI adds information not present in the source material.



Both hallucinations and omissions pose serious risks, particularly in contexts like legal work. Matt’s firm strongly advises lawyers to verify all AI outputs, treating them as they would work from a junior associate. As Matt put it, the AI is like “a first-year associate that doesn’t sleep and is always there,” but still requires careful review.




















Type of AI Error







Risk







Mitigation Strategy






Hallucination







Introducing incorrect information







Verify all outputs against source material






Omission







Missing critical information







Verify outputs and use multiple prompts to ensure comprehensive analysis






Misinterpretation







Drawing incorrect conclusions







Apply domain expertise to evaluate outputs






Over-confidence







Presenting speculation as fact







Require citation of sources for key claims






For organizations implementing AI, establishing appropriate guardrails is essential. Matt described how his firm has developed policies that provide guidance without outright prohibiting most AI use cases. They focus on education about appropriate usage contexts, data confidentiality protections, and verification requirements, creating a balanced approach that manages risks while capturing benefits.



AI Adoption in Risk-Averse Industries



The legal profession offers insights into how AI transforms traditionally cautious industries. As a product management professional, I pay special attention to adoption patterns in risk-averse sectors—when they embrace new technology, it often signals well-established value and manageable risks.



Matt shared a progression of attitudes toward AI within law firms over the past two years. Initially, many leaders experienced fear about AI’s potential to disrupt their profession. This concern was both practical and financial: AI tools represented a significant investment while potentially reducing billable hours by increasing efficiency—a challenging value proposition in an hourly billing model.



Over time, with education and exposure, these perspectives evolved into more nuanced views. Today, many firms, including Matt’s, are bullish on AI’s capabilities within appropriate boundaries. Some see it as a competitive differentiator, while others pursue AI implementation to avoid falling behind competitors.



The adoption curve among individual lawyers follows patterns familiar to any technology implementation. Matt observed that his organization has moved beyond early adopters and is now entering the early majority phase. While some users try AI briefly before abandoning it, those who integrate it into their workflow show steadily increasing usage over time.























AI Use Case in Legal







Description







Value






Content summarization







Condensing contracts, briefs, proceedings, and statutes







Saves time on document review






Synthesis







Combining information from multiple sources







Creates comprehensive understanding






Drafting assistance







Generating initial document drafts







Accelerates document preparation






Strategy brainstorming







Exploring alternative approaches and counterarguments







Enhances case preparation






Provision analysis







Identifying favorable/unfavorable contract terms







Improves negotiation position






What particularly impressed Matt after 25 years in the AI space was how dramatically the technology has evolved. The systems he worked with 15 years ago were fragile, expensive, rules-based, and easily broken when applied to adjacent use cases. Today’s models understand language nuance, adapt to specialized terminology, and apply reasoning to novel situations—capabilities previously thought to require years more development.



For product managers serving risk-averse industries, this evolution suggests several insights: emphasize verification and human oversight in your AI implementation, focus on specific high-value use cases with clear ROI, and recognize that resistance often transforms into enthusiasm as users experience benefits firsthand. The legal industry’s journey provides a roadmap for introducing AI into other conservative sectors, from healthcare and finance to government and education.



The Future of AI in Product Management



The most successful AI implementations in product management will be those that seamlessly integrate into existing workflows. Matt and I agreed that transparent integration represents the next frontier for AI tools, moving beyond standalone applications to become embedded features within the systems product teams already use.



This parallels our experience at LexisNexis, where we worked together on a product that integrated new capabilities without requiring users to change their behavior. I expect platforms like ProdPad to succeed by making AI assistance transparent and aligned with users’ natural work patterns.



By contrast, Microsoft’s Copilot approach in Office applications often feels disconnected from the actual workflow. As I mentioned to Matt, I frequently close the Copilot prompt when opening Word because it feels like an extra step that interrupts my process rather than enhancing it.



Our conversation also touched on recent developments in AI democratization. We discussed Deep Seek, which enables running sophisticated AI models on relatively inexpensive hardware—from Raspberry Pi devices to modest servers. This trend capability allows organizations concerned about data privacy and security to maintain complete control over their AI systems and data.



The future of AI in product management could include an AI Internet of Things


Matt predicted this will lead to a bifurcated market: cutting-edge models will continue to require substantial computing resources, while slightly older generations will become commoditized and available for edge computing applications. He envisioned an “AIOT” (AI + Internet of Things) future where smart devices incorporate local AI processing.























Future AI Trend







Impact on Product Management






Workflow integration







Reduced friction in adoption and usage






Local AI models







Enhanced data privacy and security control






Edge computing AI







New product possibilities with embedded intelligence






Democratized access







More accessible AI for smaller teams and organizations






Specialized fine-tuning







Tailored models for specific product domains






Beyond technical advancements, Matt expressed enthusiasm about AI’s potential social impact. He’s personally focused on applying AI to health and climate challenges. He noted that while large pharmaceutical companies are exploring AI applications, there’s tremendous untapped potential for nonprofits, NGOs, and small mission-driven organizations to leverage these tools.



This social dimension presents an opportunity for product managers to apply their skills beyond traditional business contexts. As AI becomes more accessible, product professionals can help mission-driven organizations integrate these capabilities into their workflows, potentially creating outsized impact through enhanced efficiency and effectiveness in addressing critical social challenges.



Key Insights for Product Managers



Throughout our conversation, several actionable insights emerged that product managers can apply immediately to their work with AI. We discussed the need to reframe executive demands for adding AI into customer-focused questions.



Matt and I both encountered situations where leadership teams push for AI integration without clarity about the specific value it will provide. Too often, senior executives make broad statements like “we need to add AI to what we’re doing” without understanding what that actually means for products or customers.



As product professionals, our responsibility is to translate these directives into customer-oriented questions: How can we enhance our product’s value using AI to solve problems customers actually care about? Will AI help solve these problems faster, better, or more comprehensively? This reframing helps ensure AI serves genuine customer needs rather than becoming a superficial feature.



Another opportunity Matt highlighted was AI’s ability to process unstructured data. He noted how frequently we encounter friction in our daily lives—retyping information into forms or manually extracting data from documents only to re-enter it elsewhere. These pain points represent prime opportunities for AI-enhanced products.























Customer Pain Point







AI Application Opportunity






Form completion







Auto-extraction of information from existing documents






Data transcription







Converting formats without manual retyping






Information synthesis







Combining data from multiple sources automatically






Content transformation







Converting between visual and text formats






Pattern recognition







Identifying trends in unstructured information






Matt shared personal examples of AI’s potential, including using it to solve scientific puzzles from Scientific American and helping his daughter understand idioms in her schoolwork. These seemingly simple applications demonstrate how AI can remove friction points that we’ve previously accepted as unavoidable.



Conclusion



The integration of AI into product management isn’t just a passing trend—it’s fundamentally transforming how we research customer needs, prototype solutions, and create value. AI has evolved from a specialized, experimental technology to an essential tool in the product manager’s toolkit. The question is no longer whether to incorporate AI into product development processes, but how to do so most effectively.



As you incorporate AI into your product management practice, remember that the technology itself is just a tool. The true value comes from how you apply it to understand customer needs, solve meaningful problems, and create products that improve people’s lives. By focusing on these fundamentals while embracing AI’s capabilities, you’ll be well-positioned to thrive in this new era of product innovation.



Useful Links





Innovation Quotes



“The best way to predict the future is to create it.” – attributed to Alan Kay and Peter Drucker



“The way to get started is to quit talking and begin doing.” – Walt Disney



Application Questions




  1. How could you integrate AI as a brainstorming partner in your current product development process? What specific activities (customer research, feature ideation, user story creation) might benefit most from this partnership approach?


  2. In what ways could your team use AI to accelerate prototyping without sacrificing quality? How might this change your current approach to validating product concepts before full development?


  3. How could you transform executive requests to “add AI” into customer-focused initiatives? What specific customer problems in your product area could AI help solve more effectively?


  4. How could your organization balance investment in specialized AI tools versus leveraging general-purpose AI models with custom prompts? What ROI metrics would you use to make these decisions?


  5. How could you ensure AI integration enhances rather than disrupts your existing product workflows? What would seamless integration look like for your specific product and team?



Bio




Product Manager Interview - Matt Coatney


Matt Coatney is a seasoned C-level AI and product executive with 25 years of diverse experience. His expertise includes: artificial intelligence, business growth, and product development. Matt has also supported a wide range of industries such as manufacturing, media, law, life sciences, government, and finance. His client list includes some of the largest, most well-known organizations in the world, including Microsoft, IBM, the Bill and Melinda Gates Foundation, Pfizer, Deloitte, HP, and the US government.



Matt writes and speaks frequently on technology and product topics. In addition to a TED talk and keynotes, his work has been published by MIT, HarperCollins, and O’Reilly and has appeared in books, journals, and international conferences. Matt’s latest book is The Human Cloud: How Today’s Changemakers Use Artificial Intelligence and the Freelance Economy to Transform Work, with Matthew Mottola.



Thanks!



Thank you for taking the journey to product mastery and learning with me from the successes and failures of product innovators, managers, and developers. If you enjoyed the discussion, help out a fellow product manager by sharing it using the social media buttons you see below.



Source