How we decide who rules the roost.
Every ranking on The Roost comes from one measure: how often AI assistants find, trust and choose an agency when real buyers ask. This page is the whole rulebook, dated and versioned, so anyone can check our work.
Five rules. No exceptions.
A rule only changes with a new method version, announced at least 30 days ahead and logged on this page.
Nobody pays to rank
No fee, plan or partnership changes where an agency sits, its score or its range.
One method for everyone
Every agency in a market is read with the same questions, the same assistants and the same maths, whether it is a RocMode customer or not, claimed or not.
Dated and versioned
Every score shows when it was read and which method made it. Scores from different methods are never shown as movement.
Claims show receipts
What an agency says about itself is shown with its source and status. Claims shape how much a buyer can trust a profile. They never change rank.
Anyone can ask for a correction
If we got a fact wrong, tell us. Accepted corrections are fixed and logged in public.
An industry, in a place.
Buyers in Brisbane ask different questions from buyers in Perth, and AI answers them differently. So every ranking is for one industry in one place, with its own set of buyer questions. An agency can rank in more than one market: where it is based, and where it serves. Profiles say which is which.
Example. Live markets will be listed here as they publish.
From a buyer's question to a rank.
Build the question set
The questions buyers in that market really ask AI, at each step of choosing: noticing a need, learning, comparing, choosing and buying.
Ask the assistants
Each question goes to the AI assistants buyers use, starting with ChatGPT, Claude, Gemini and Perplexity, and is asked more than once, because answers vary from run to run.
Read every answer
For each answer we record which agencies were named, whether each was described correctly, whether its claims were backed, and who was recommended.
Score each agency
The readings become a score per buyer moment, then one score per agency with a range that shows how certain it is.
Publish
The ranking goes live with its date, method version and the questions asked. Every answer stays on file.
If an assistant can't be reached during a read, the result says so and the range widens. We never fill a gap with a guess.
Seen isn't selected.
We follow a buyer up seven stages, from hearing a name to staying a client. Found decides whether AI names an agency at all. The other six make up Quality: how well AI carries the agency once it's named.
| Stage | What AI has to do | Weight |
|---|---|---|
| Known | Buyers already search for the agency by name. | 10% |
| Found | Name the agency when it names agencies for a buyer's question. | Gate |
| Answered | Describe what the agency does, correctly. | 15% |
| Believed | Back the agency's claims with proof it can point to. | 20% |
| Preferred | Recommend the agency over the alternatives. | 30% |
| Bought | Point the buyer to the next step with this agency. | 15% |
| Kept | Ratings clients leave after hiring, from review sites. | 10% |
Quality weights sum to 100%. A stage with no reading is left out and the others are rescaled. It is never scored as zero.
One number, 0 to 100.
Found = share of answers naming any agency that name this agency
Quality = 0.30·Preferred + 0.20·Believed + 0.15·Answered
+ 0.15·Bought + 0.10·Known + 0.10·Kept
Moment = √(Found × Quality)
# per agency
Score = weighted mean of its moments, with a range
Answers that name no agency at all, such as general advice, are open ground. No agency can win them, so they are left out of Found rather than counted against anyone.
Nothing you can buy.
Every claim shows its receipt.
Profiles show what an agency says about itself, from its own site and from RocMode. Each claim carries a status and the evidence behind it, quoted word for word with its source.
Think we got it wrong? Tell us.
Tell us what's wrong
Name the fact, the page and what's correct. Agencies, buyers and anyone else can ask.
We check it against the evidence
We look at the sources and the answers on file. A rank is only re-read at the next scheduled read, the same as everyone.
The fix is logged
Accepted corrections are fixed and listed here with the date.
Read on a set calendar.
Each live market is re-read on a published schedule, and movement is shown with its reason. The calendar for Australia's first markets will be posted here when they go live.
The referee doesn't play for either team.
The Roost is powered by RocMode, which also sells brands and agencies the plan and tools to improve their Selectability. We say that plainly. RocMode customers are scored by the same method as everyone else, and their gains only show up after the next independent read.