Skill Health

Overview

How might we…tell Claude, and the person using it, which instructions in an installed skill are no longer true, before Claude follows them?

Problem

A skill is a text file of instructions Claude follows: install commands, flags, imports and links. The software it describes keeps changing and the file does not. gcp-cloud-run, one of the skills I use most, tells Claude to push images to gcr.io. Container Registry stopped accepting new images on March 18, 2025, so whether that push works now depends on the project's Artifact Registry setup.

On September 27, 2026, I checked 12 skills for fast-moving software, including Next.js, Supabase, Stripe and Cloud Run. Every file still matched its repository byte for byte, so an update would have changed nothing, yet 10 of the 12 had an instruction that fails or is deprecated.

Update tools answer one question: is there a newer copy? Here the answer was no, and the instructions were still wrong. Skill Health asks a different one: is this instruction still true?

Checking the instruction, not the copy

Each instruction you could act on goes to one agent, which compares it with the current documentation of the software it is about. A second agent then tries to prove each finding wrong before you see it. On October 9, 2026, across the 79 skills I had used by then, 104 failing and 36 deprecated instructions held up, and the second agent threw out 15 of 155 findings. One of the 15 was a correction I had written myself: it said npm install -g supabase fails, and since Supabase CLI 2.99.0 it works.

The check runs on your own Claude plan with no API key. It asks which skills to check, and what that will cost, before anything runs.

Telling Claude, not only the person

A list of problems helps only if someone reads it, and Claude follows a skill whether or not I did. So at the start of a session, Claude gets each correction as one line: the skill, the instruction, what is true now and a link to the evidence.

ai-sdk says import { useChat } from "ai/react". The ai/react export was removed in AI SDK 5.0. useChat now comes from @ai-sdk/react. Evidence: https://ai-sdk.dev/docs/migration-guides/migration-guide-5-0

Claude gets up to five, from the skills used most. On my Mac they come to about 650 tokens.

The plugin does the work, the app says when

The first version had three ways to find problems: corrections bundled in the app, a command in Claude Code, and a check inside the app with your own API key. That spread one job across three surfaces and asked people for a key. I removed the key and the in-app check and moved the corrections into the plugin. Claude Code now does the work, and the menu bar app has one job: show a dot when a skill you use needs a fix, and list what needs fixing, what is due for a check and what has a newer copy.

The menu lists skills that need fixing, skills due for a check and skills with newer copies, each with its counts and how often you used it.

Setup is the only window. Each row checks itself: the folder row shows how many skills it found and how many were used, and the plugin row turns to Connected.

Decisions that shaped it

You approve every change to a skill

Fix goes through one finding at a time. You choose to fix the line, set it aside for 30 or 90 days, draft an issue for the skill's author, or name another skill that covers it. Claude Code's permission prompt shows the exact change before it is written, and every replaced file is backed up, so /skill-health:undo can reverse a fix or an update while the files are as Skill Health left them. A hook refuses the apply command when permission prompts are turned off. It reads the command's text, so it is a safeguard, not a sandbox, and SECURITY.md says what it does not cover.

Updates never write over your edits

For skills installed with openskills, a newer copy is taken only when yours is unchanged on this Mac. Fixes are written only inside ~/.claude/skills.

Copies already checked cost no tokens

Known corrections match your skills by exact file contents, so a copy identical to one already checked shows its wrong instructions without a new check. There are 89 today, for 62 checked copies of 61 skills from three public collections.

Say what it cannot tell you

A finding is judged against the software as it is today, not the version a project is set to use, so a project that stays on AI SDK 4 can still use ai/react. Accuracy has been measured only by the second agent so far, not by people reviewing a sample, and all the evidence comes from one person's skills. The README lists these limits, and every number carries its date.

Shipping it

Open source under Apache 2.0, at version 0.6.1 and marked early. The plugin installs with one command and needs macOS 27 and Claude Code. The Mac app is optional and built from source, with no signed download yet. It has no third-party dependencies.

claude plugin marketplace add mvoshell652/skill-health && claude plugin install skill-health@skill-health