For some stupid reason, I thought this was going to be a short report month. Instead, a wild Bubba appeared!

So what I’m going to do is publish two reports in quick succession here. This one covers the new position and a bunch of things that I know that I want everyone applying for that job to have access to. I’m also going to walk through the changes to the Headmaster position, which I think got lost in the excitement around the new position.

New Council Position: Emissary

So I think it’s worth talking a bit about this job, because when Bubba pitched it to the Council and the Consuls, I think there was a lot of confusion about what the Emissary is supposed to do. I think the major assumption is that it’s primarily a recruitment position, but that’s not entirely accurate.

To back up a bit, we have two streams of new members with very little overlap. The first group are the people we usually call “recruits” or maybe “direct recruits.” These are people who had an existing relationship with a DB member who brought them into the club. So we have a lot of family members, a few coworkers, and a steady trickle of people from other clubs, usually but not always the Emperor’s Hammer or Bravo Fleet. We tend to be pretty good at retaining these people. But we tend to not get a lot of them, and it’s a relatively small pool to draw from.

We also have the group we call “random joins,” “round-robin joins” (after how they’re assigned to clans), or something like that. These are people who find the club somehow and sign up. And our retention rate for these guys is very bad, and it’s very bad in a multifaceted way, and when we increase how many of these people we get, the retention rate tends to get proportionally worse. We tend to yell vaguely at the clans about retaining these people, but the reality is that we lose most of them before they even interact with a DB member, and there’s not a lot that a clan summit can do about that. Worse, generally speaking we have no idea why those members, who found our club interesting enough to sign up for, decided to nope out before meaningfully engaging with us.

So I think the Emissary should work on recruitment to increase the number of signups, and they should work directly and through clan summits to ensure new members who make it to the Discord server are integrated into the community. But frankly, we don’t need a new DC position to do that. What we need is someone to identify where and why we lose potential members. We need someone to take those insights, come up with plans to fix our issues, and then implement them.

So a good example of that is roleplaying. The point of that whole project wasn’t because I like chat RP (I don’t particularly) or because I wanted Elder (lol, just lol). The point was that Gui and the Odanite summit were finding it was significantly easier to pull people in from Discord chat RP servers than other Star Wars fan communities, and meanwhile Arcona was eating everybody’s lunch in large part thanks to members they’d pulled in through Roll20 and similar tabletop-oriented groups. Both clans found it was pretty easy to run chat RP and members liked it, but at least in COU, we had a major problem because we had members joining to do chat RP, and the rest of the club was set up to do everything else. Chat RP didn’t get you promoted, it didn’t get you more CS points or credits, and the club needed to change to accommodate those members rather than expecting them to change what they enjoy doing to suit our weird legacy systems.

I think that’s ultimately what we’re looking at. The task is to move from “I think the problem is…” to “We know the problems are…” and to coordinate with the entirety of club leadership to address them, whether that’s changing how we do advertising and outreach, change how the site is laid out, changing the new member process, changing how we structure and present our core activities, or whatever else you find. You can’t have plans for doing those things yet because we don’t know what things need doing.

The Emissary Data Dump

So let’s talk about what we do know and what you can plan for. I’ll just say up front that I think the most important data is not something I can give you. The Emissary is going to need to target specific communities of potential recruits and, whatever else they do, they’re going to have to talk to those people and find out what they think about the DB and why they are or are not interested in sticking around. That’s not going to be a neat quantitative data set but I do think you should ask people somewhat systematically and then write that shit down so we have a big set of responses and not just gut hunches based on what you think you remember ?? people said.

Anyway, what we do and don’t know. This folder should give you view access to five files. You can’t filter or do anything with these documents because they’re view-only. So I’d suggest making your own copy, whether you do Excel Sheets magic on them or just read through the responses to do a gut check.

• 2023 Year-to-Date is the new join data for 2023 so far, straight from the export tool James built into the site.

• DB-Wide Analysis, Q2 2023 is a document I copy/pasted into Yammer (Council and CON/PCON Chat) in July. It links to its own spreadsheet of raw numbers and covers from April 2022 through June 2023.

• Exam Results has a data dump from Essentials 101 and the Advancement Survey. Thank James for this one.

• Google Analytics is Google Analytics. Please read the explanations and caveats for everything, but especially for this one, especially especially if you’re familiar with Google Analytics.

• New Join Data (from Septemberish 2022) is a spreadsheet I made for my HM application that has some retention info from 2017 through about a year ago.

I’ll go through what the hell all that gibberish even is, but first there’s one more thing that I think you’ll need. This is our join procedure. It’s a nice, step-by-step overview with screenshots of what our new sign-ups and NV/JMs see. Unless you’re brand new, it might have been a long time since you saw any of this. If you’ve been here more than a few years, there are steps you’ve probably never seen. So you should really soak this up and, while you’re at it, thank James for building this and giving Bubba and I the link to it several times a year because we’re idiots and keep forgetting where it is.

If you have questions, if you would like some data pulls (through the tool, if you want JSON or something ask James), whatever, please feel free to reach out to me. I don’t have Discord analytics, unfortunately, because Discord won’t turn them on until you have 400 members and we’ve only got 380 or so.

2023 Year-to-Date

There’s a tool James built into the site that queries the database and spits out nice, human-readable info, and this is the pull for new joins from 1/1/2023 through today, November 15. Most of this is pretty self-explanatory but here are a few notes.

Path:
This is no longer a thing as of last year, although you’ll notice members are still prompted to select Light or Dark when they join. That’s because new joins who don’t have a recruiter listed get assigned to clans based on that choice. That’s something CONs work out with the DC and currently the breakdown goes:

So when the next rando signs up, if they select Dark, they’ll get placed in CSP, but if they select Light, they’ll get placed in COU. If you think that's a stupid way to do it, fair enough, but keep in mind the Cov says units are entitled to a fair share of random joins.

Join Date:
It’s month-day-year, sorry non-Americans, you can make a copy and set your own date format.

ToI Progress:
This is the Trial of Identity, which you can see starting in screenshot five in the join documentation. This is what triggers the promotion from NV1 to NV2 and members get assigned to a clan only after completing the ToI. “8/6” means they completed it, anything else means they didn’t.

Recruited:
What this actually means is if the member listed a recruiter in the sign-up form or used a recruitment link. If they did, they’re “Recruited” and if they didn’t, they’re “Round-Robin.” In reality, some Round-Robin joins were brought to us by a member while a lot of Recruited joins were hit up out of the blue by a stranger with a join link (hi Gui).

Discord Authenticated:
If you remember, when you first join the DB discord server, you get plopped in #welcome and have to copy/paste a code from the site to link your Discord account to your DB account. Members who did this are TRUE, members who didn’t are FALSE.

DB-Wide Analysis, Q2 2023

I copy/pasted this into Discord along with screenshots, so pardon the references to screenshots you don’t have and the raw Markdown. I’ll caveat that these are my own dumb thoughts and opinions.

This links to a spreadsheet called “AVG Q2 2022 - Q2 2023 Report.” This is what I try to do for club leadership every quarter using that report I shared part of in the last doc. There are three tabs: Unit Metrics (AVG), Unit Metrics (SUM), and Retention Date (SUM). The AVG one is averaged across the seven clans, the SUMs are the totals for the clans and the Council. What you probably care about are the top portion of the second tab and the third tab.

Most of the gray box in the first two tabs is either self-explanatory or labeled. “Random Joins” and “Direct Recruits,” if you haven’t guessed already, are the people listed as “Round-Robin” and “Recruited” in the raw export.

The Retention Data tab is an ugly thing I added last minute a few quarters ago and never prettied up. Basically, you have how many new members joined in a given quarter on the left, and as we go from left to right, you see how many of those members were still around. (“Around” means they showed up in the data export, so if they received a medal or did a comp; they could still be clanned if they show up for AWOL checks.) So in our first row, we have 16 members who joined in Q2 2022, 5 of them were still around in Q3, and so on, until only 2 were left when I pulled that data on 4 July 2023. I have retention rates on the right there. “Rate excl. Q2 2023” is what percent of members from Q2 2022-Q1 2023 are still around. I broke that out separately because a lot of “retained” members from the most recent quarter haven’t been around long enough to AWOL out, but they’re already effectively gone.

The top batch is all new accounts, then Round-Robins in the middle, then Recruiteds on the bottom.

Exam Results

This has two tabs: Essentials 101 and Advancement Survey. Along the top, you have the questions, and then member responses.

Essentials 101: Welcome to the Brotherhood launched in January as a directory to Brotherhood Fundamentals courses and the exam basically just asks why they’re here. We have 114 responses and the key caveat is that most of them are not actually new members. The responses are, as far as I can tell, presented in a totally random order. Maybe one day I’ll have a version of this with when the member joined and when they took the exam, but this is what James gave me this morning, so this is what you get.

The Advancement Survey responses go back to 2013, although for whatever reason we only have 448 of them. The data is… messy. We do have older members doing this, but generally the survey is written for a member in the later JM ranks, but the people who actually take it tend to be very new joins who are blitzing through the entire Fundamentals department during their first few days in the club. So there are a lot of questions about clan summits and the highest positioned member they’ve interacted with, and the people answering them don’t actually know what positions the people they’ve talked to are in. But again, it’s what I’ve got so here you go.

I’ve had vague plans to present a cleaned up version of this in a year-in-review report but never got to it, so if you get the Emissary job, you should bully me about that.

Google Analytics

Hoo boy.

Ok, so, the first step of our new join process is the one we understand the least, namely everything that happens between someone getting to our site and becoming a member. We don’t really know how many people find the site, what percent of them sign up, or how organized recruitment campaigns impact those numbers. (We do know that advertising, whether volunteer or paid, increases sign-ups.) Google Analytics is the tool that would normally tell you all that information, and it’s very good at it. But I need to caveat the shit out of this, gentle reader.

First caveat: Our data only goes back to mid-May. We had some other analytics data predating that, but when Google shut down Universal Analytics and moved us to GA4, we basically started over. Or at least, I don’t have pre-existing data.

Second caveat: we don’t have real Google Analytics. In order to use that, we would need a big ugly GDPR banner plastered on the site and Evant, Bubba, and I decided that was probably not worth it. You, our future Emissary, may or may not want to revisit that. But the way this tool is built to work is that it should track unique visitors, which requires some technical fingerprinting and light stalking that we don’t do in our implementation. So when GA4 talks confidently about “Users” vs “Sessions” and such, you’re gonna wanna take that with a big grain of salt.

Anyway, I’ve copy/pasted our stats since May across four tabs.

User Tech:
This is basically four different ways of saying that a light majority of our traffic is from phones and the rest is almost all desktop browsers.

Users by Country ID:
So first off, I don’t know what GA4 thinks a User is. I assume, given our configuration, it’s unique combinations of IP address and user-agent string, but that’s honestly a guess. If I’m right, then I’m probably several users a day, because not only do I visit the site from both my phone and my desktop, sometimes I’m using a VPN and others I’m not, and if I’m on mobile data, then my phone’s IP address will change often throughout the day. So the numbers are probably garbage.

Some of this makes sense to me, like the majority of our traffic likely is from the US, UK, Canada, and Germany. But I’m skeptical that we have more visitors from Russia and China (as far as I’m aware, we don’t have members in either country) than we do from, say, the Netherlands (Hi James! Hi Aylin!). Maybe people just have poor taste in VPNs, but since we know we have a lot of bot sign-ups, I suspect a lot of traffic across the board is bots, that those aren’t evenly distributed by country, but that we don’t have any way at present to differentiate bots from people.

Per Google, a Session starts when a unique user hits our site, and stops when we haven’t seen them for 30 minutes.

Traffic Acquisition:
Again, I don’t trust the raw numbers entirely. But this is a breakdown of how visitors get to our site, and probably at least in the right ballpark. Maybe. I hope.

This breaks sessions down by “session default channel group” and I’ve copy/pasted what those actually mean in column L there.

Pages and Screens:
This is traffic to individual pages, annoyingly enough presented as titles only and not URLs. Most of this makes sense: the main page is most popular, with the SA, Wiki, ACC, and Roster main pages and the competition overviews for GJW XVI and Pro Bowl VII rounding out the top. We also see a lot of traffic to individual dossiers for reasons that aren’t entirely clear. Why is the fairly obscure wiki article for the Lotus, which hasn’t been relevant to the club plot in five years, our 8th most popular page and most popular individual wiki article?

¯_(ツ)_/¯

New Join Data (from Septemberish 2022)

This is taken from a supplement to my HM app last year (yeah, I know), and I haven’t updated it since whenever I made it, and I don’t remember off the top of my head what the data cutoff was, but it looks like all new joins from 1 Jan 2017 through 30 Jun 2022?

Anyway, two tabs here. “Dashboard” is stuff, “Joins” is just a copy/paste of the then-current data pull I worked off of. Dashboard shows (top to bottom, left to right, apologies to the colorblind):

New Joins by Year (Blue):
For each year, how many new accounts we had and how many were still in a unit. Keep in mind 2022 is only half a year, so the numbers ended up being higher.

Current Rank of Members by Year of Recruitment (Red):
I switched axes from the last chart, sue me. For all the joins, this was their rank as of whenever I did this.

Current Location of Members by Year of Recruitment (Yellow):
Are they in the Rogues, or are they in a unit? If so, which? Off the top of my head, I’m the dossier-youngest DCer, so none of the Council are in this chart.

Current Location of Direct Recruits (As Opposed to Round-Robin Joins) by Year of Recruitment (Purple):
The same as the last chart, only limited to recruits. This is where they are now (well, last year), not where they were recruited to.

Members Who Have Attained a Rank of at Least (Green):
So this and the pretty bar charts are laying out how far people got. This is good for finding our bottlenecks. So we had 1495 “new members” in four and a half years. That’s a lot! But only 993 completed the Trial of Identity and validated their email account to get their NV2 promotion. So that’s about a third of new accounts that didn’t make it far enough for a member to break out the welcome wagon.

Of those 993 accounts that made it to NV2, 436 got promoted to NV3. This was pre-XP, so that promotion could be unlocked several ways, including doing any SA exam, joining Discord/Telegram, submitting to a competition, basically anything that demonstrated you were a real person. So out of 1495 new accounts, we’ve already lost 71% of them.

About half of the NV3s stopped there. We had much smaller (but still pretty large) attrition rates through the JM ranks–remember, this was pre-XP so it was all checklist based promotions. I know of several members who’d rogued out as JM2s or JM3s and were knighted when XP launched.

Overall, only about 5% of the 1495 new accounts made it to Knighthood, which is traditionally a major milestone, although XP really cut into the significance of that promotion.

New Council Position: Headmaster

So, you might be thinking, Arch offloaded all the new member duties, huh?

Yes! But also no.

My updated job description is here.

Long story short, my mission now is more or less what it’s been for a while now: knowledge management.

Short story long, those of you looking at the Advancement Survey responses will see the word “overwhelm” and variants 46 times across member responses. You’ll see “It’s just a lot to take in all at once” and “It is a lot of information to consume” and “there is a lot of info here!” and “Whn i first joined, i had no idea what i was getting into” and such. You’ll see comments like:

The most confusing thing I have encountered has been the Wikis being out of date/having conflicting or missing information. It made things more challenging

Don't be afraid to ask questions, in my experience the leadership is...very patient...I asked a lot of questions…

A lot of new stuff like possessions, credits, loadouts, snapshots. Just to take part in a fiction comp for the GJW I had to figure out the snapshot thing. The Shadow Academy is a great way to learn, but not sure the average new person will put in the effort to learn that way. Really need that mentorship at the beginning for new members. Just items like a storyline are intimidating, this hole Collective thing has been ongoing but to jump in the middle makes you unsure to take part. So leadership need to be very vocal and ask people if they have questions and some won't just speak up they will disappear.

As someone with ADHD, I have a lot of issues when there is a lot of information. If I'm going to pass the test, the source information can't be 10000 words long as in a wiki because my head can't handle it, but everything else I've looked into appears to be acceptable and easy to access. I think that the early ranks in and through journeyman should in fact have many of those basic exams from the fundamentals in them so even if you joined and all ready doing game stuff or graphics knowing that you have taken them means you'll have a better understanding.

And so on.

So first, we have a lot of information we present because we do a lot of things. We’re a multi-platform gaming club, we’re a fanfic club, we’re a chat roleplaying club, we do art and puzzles and such, we have a Discord server with hundreds of members and dozens of channels and special events, we have almost 30 years of accumulated history and four hostile factions and seven player factions and like two dozen officer positions whose names have no relation whatsoever to what they do. So yeah. It’s a lot. And it hits new members like a freight train.

Second, we’re bad at presenting that information. Some things are documented right in the menu on the site. Some things are documented in the SA, others aren’t, and a lot of SA courses have inaccurate information. Some things are on the wiki, others aren’t, very often you’ll see similar pages on the Wiki with conflicting information on the same topic. So if you’re a member, and you’re trying to remember how to set up an RP session, where do you look? The SA or the Wiki? The Server Guide on Discord? The pinned messages in #rp-lobby? And here’s the kicker: if you find information in one of those places, do you actually trust it? If you give up and just ask somebody, how? PM Wally? Ask the huddled masses in #rp-lobby? Email the staff? So even established members struggle with this constantly.

So making the Brotherhood, in all of its weird, multifaceted, convoluted glory something that a new, returning, or even established member can easily wrap their head around? That’s a big task. I figured I’d start by making the SA a current, easily usable, first-stop source for guides on anything you’d need to know to engage with any activity in the club, and that was already a lot.

But it was also never going to be enough. I or a future HM was always going to have to engage with James on the site. I or a future HM was always going to have to work with the Wiki Tribune and the Council to triage the Wiki, to curate a set of consistently maintained documentation that stands apart from articles a House Tridens AED wrote years ago that haven’t been touched since. It was always the HM’s task, now it’s also the HM’s job.

I and my staff aren’t going to be on the hook for being the club welcome wagon anymore, but we’re going to remain neck deep in new member experience, along with returning member experience and new feature rollout.

So for the time being, Bentre reports to me instead of Bubba, but otherwise he’s still the Wiki Tribune and he’s still doing his thing. This happened quickly and we haven’t had a chance to talk through how that’s going to work. I don’t have immediate plans or big announcements. We’ll get there.

Similarly, Kanal reports to me now instead of Bubba, because I can kick and scream about how running trivia comps isn’t the HM’s job all I want, and it still makes sense. Kanal used to work for me as M:HM, doing basically the same job, so I don’t expect us to reinvent the wheel.

If you have Wiki issues or questions, Bentre is still your first stop. If you have trivia issues or questions, Kanal is still your first stop. If, for whatever reason, you need a second stop, then I’m your bird.

Closing Remarks

I have a few SA-specific updates to release, including a new course, but this was enough to digest without making everyone weed through recruiting and retention stuff and vice versa. I’ll just drop the spoiler that the starship exams are a little shorter now and get back to you in a couple days.

As always, if anyone sees any errors in the course notes or exam questions, please submit them through the form. Scrolls of Foundation are a nutritious part of a healthy XP diet.

And, of course, I am always available at [Log in to view e-mail addresses] or Archenksov#2230.