[ALAC] ICANN and measurement of "consumer trust" -- yeah, right

[Evan Leibovitch]

GNSO's Consumer Choice, Trust, and Competition Working
Group<http://gnso.icann.org/en/group-activities/consumer-trust-wg.htm>,
on which I have sat since its inception, was an ill-conceived 2010
direction from the Board to identify metrics that could be used to
demonstrate that the new gTLD program provides benefit to registrants and
end-users.

(Of course, in any sane organization such measurement would have been done
*before* actually determining need for gTLD expansion, but I digress...)

So this working group was tasked with developing metrics that would --
after the fact -- serve to demonstrate that ICANN has justified in going
through the expansion. This working group has finished its final
report<http://gnso.icann.org/en/issues/cctc/cctc-final-advice-letter-05dec12-en.pdf>,
and will be submitting it to the GNSO council soon.

If endorsement of the report comes to ALAC for consideration as-is, I will
oppose its endorsement and I ask for other ALAC members to join me in its
rejection.

To be sure, there are some possibly-useful metrics for registrants proposed
-- such as whether trademark takedown claims will be higher with new gTLDs
than with the existing ones. But it is already noted that some of these
metrics will be difficult or expensive to implement, and may either be
unsupported or actively impeded by registrars if not required in the RAA.

The biggest metric from and end-user point of view -- a survey of whether
end users around the world will encounter less cybersquatting, confusion,
park sites, malware, etc -- is crammed into a single item (1.4), with a
proposed "picked from thin air" budget of $100,000, and acknowledgement
that even the creation of the survey's questions will be challenging.

More important, though, is what is missing from the recommendations.

Much like ICANN exists in its own bubble shielded from the world, there are
no metrics that go outside the scope of the TLD expansion itself. Nothing
is proposed to determine whether people are even preferring domain names to
get to Internet content over alternatives such as search engines, QR codes,
domain-name-shorterning services or social media portals and shortcuts(*).
That is -- IMO one of the most important metrics -- whether the gTLD
expansion is turning people away from the DNS itself, towards friendlier
and less confusing ways for end-users to reach their Internet destinations.

I have been presenting this issue to the WG from day one. It was deemed out
of scope. Though wording of mine indicating the need for greater scope was
put in the report's preamble, it is highly unlikely that ICANN will act
upon anything not called for in this report.

Still, I was willing to support the recommendations as being better than
nothing. In October a "final"
statement<http://forum.icann.org/lists/gnso-consumercci-dt/bin58KclzeJuJ.bin>,
that had the consensus support of the working group, was produced and
prepared for the GNSO. (The link has a "bin" extension but opens as an
msword document).

At that point, a member the registry constituency -- who had not been part
of the consensus, or for that matter ever participated in the working group
up to that moment -- demanded that significant parts of the metrics be
dropped.

The metrics to be dropped eliminated "closed" TLDs from many important
measurements. They elimininated, for one thing, the masurement of whether
"closed" TLDs are providing easy-to-find information of how to report
abuse, or how domains in that TLD are allocated.

After more than a year of hard-fought consensus, during a call which I was
unable to attend, these demands caused changes to many specific metrics
that now renders them useless. What were once objective audits (2.1 and
2.2) are now undefined "measurements of understanding", a process that is
difficult, expensive, and trivially ease to deliberately misinterpret. I
don't believe that measuring public attitudes should distinguish between
"closed" and "open" TLDs whose domains are all publicly accessible. If a
domain in a new TLD is the source of malware, spam or any kind of end-user
confusion, does it matter to the end user whether that domain is open or
closed?

Anyway, what is now being presented to the GNSO is a "new" final report
that incorporates these demands. My protests about both the content changes
and their last-minute inclusion have been met with personal attack and
threats that the Registry SG would scuttle approval of the Report unless it
incorporated their demands. So much for consensus. The registries have a
vote at GNSO and we don't. The report includes a letter I send in August in
its appendix, but refuses to note any minority dissent to the last-minute
changes to its "final" report.

One might be tempted to say, "having some metrics is better than having no
metrics", especially having invested substantial personal capital in
creating them. But I suggest otherwise -- that the newly-created biases in
the questions, along with the changes from audits to uselessly subjective
measurements, will lead to results that will be embarassing to ICANN while
demonstrating nothing after much needless expense -- in other words, worse
than if nothing is done. In its non-consensus revised report, the WG
has pacified the domain industry's refusal to measure things whose results
would be embarrassing, at the expense of ICANN's own credibility to measure
itself.

There are two other At-Large members on this WG who did not support my
objections -- Cheryl and Olivier can certainly speak for themselves why
not. So perhaps the accusations were correct, and I stand alone in my
assertion -- that these metrics in their current form are just a PR stunt
so that ICANN can justify its TLD expansion after-the-fact. In any case, I
ask you to consider the issue and understand why I refuse to endorse it.

The main saving grace of all this is that, in the scheme of things, it's
not very important. If ICANN botches how it measures its success, others
(with non-ICANN-positive biases) will be happy to step in to fill the
void.It's not even policy (which makes me wonder why the Board gave this to
the GNSO in the first place).

If anyone is interested in working with me on an alternative ALAC statement
 on how ICANN can demonstrate the improvement of consumer trust in its
actions, please let me know. Otherwise, I'll explain my objection, vote
appropriately if endorsement comes to a vote, and move on.

- Evan

(*) yes, QR codes and shortening services still require the DNS, but they
don't need a TLD expansion either, as they could work with obscure or
third-level domains in existing TLDs.