Shopify Engineering·Jun 15, 2026·11 min read62Teaching Sidekick to say no: automated data curation with LLM judge consensus (2026) - ShopifyOur Sidekick team used LLM judges to automatically curate training data and teach domain-specific models how to refuse impossible queries.Distributed SystemsLLMsRead original on shopify.engineeringScore breakdownTechnical depth52Practical value75Originality100Writing quality100Source reputation86Recency1External engagement0On-site engagement10NayaRead shows a summary and links to the original article. The full article is hosted on shopify.engineering.