Signup Systems Survey

[PurkkaKoodari]

Since the system is open-source and will be offered for all to use, I decided to survey opinions around signup systems. The survey asks the following:

• Background info:
  • Guild/organization
  • Activity level in events
  • Which event signup systems you/your organization use
  • Whether you are in a relevant committee for event organization or system administration
• Opinions on issues that commonly affect popular signups
• Opinions on potential solutions to those issues (like those proposed in Add pre-queuing to events #18 and Add entry randomization #19)
• Whether your organization might be interested in this project
• Other comments

The survey is available here: https://forms.gle/5KzvYkWVSiGw9D1B8
After a few days, I'll post a follow-up with analysis of the results. The comments here will be locked until then - please post relevant comments in the survey itself.
The results are posted - feel free to discuss!

[PurkkaKoodari]

Some statistics on signup systems by organization. There are a few clear groups:

• Minor forks of Athene's system
• Fully custom implementations
• WordPress plugins

Organization	Systems used (may be incomplete)
AYY	lomake.ayy.fi, kide.app
KY	kide.app
TF (Teknologföreningen)	?
Athene	Athene Ilmomasiina
AS (Automaatio- ja systeemitekniikan kilta)	Athene Ilmomasiina (fork)
FK (Fyysikkokilta)	Athene Ilmomasiina (fork)
Inkubio	Athene Ilmomasiina (fork)
PT (Prosessiteekkarit)	Athene Ilmomasiina (fork)
KK (Kemistikilta)	Custom (PHP)
KIK (Koneinsinöörikilta)	Custom
Prodeko	Custom (Symfony), new Node.js app coming soon
SIK (Sähköinsinöörikilta)	Custom (Next.js), Google Forms
TiK (Tietokilta)	Custom (Ruby), this repo in progress
DG (Data Guild)	Events Manager for WordPress
IK (Rakennusinsinöörikilta)	WordPress Contact Form 7
MK (Maanmittarikilta)	WordPress Contact Form 7
PJK (Puunjalostajakilta)	WordPress Contact Form 7
VK (Vuorimieskilta)	WordPress + O365 Forms based
AK (Arkkitehtikilta)	Google Forms

[PurkkaKoodari]

The survey has now been open for over a week, and promoted on #otaniemi, Tietokilta's Telegram groups, and various digi committee groups. In total, we got 100 responses. It's time for some statistics.

Answer counts do not match up perfectly: some questions were added a bit late, and a couple answers trickled in while writing this post. Notes in italics are my personal opinions.

Survey demographics

The survey started with some demographic questions.

13 responses mentioned other organizations; NuDe, Tokyo and TKO-äly were mentioned more than once.

Overall, we have a pretty even coverage of Otaniemi students. Tietokilta is of course an exception, as I posted the survey to our own groups as well - I will also compare our guild's results to the overall results in later questions.

I would also describe these values as pretty average. I don't think we should weigh the results too much here.

About half of the respondents participate in organizing events, about 10% manage organizing events, and about 20% participate in digital systems development.

Basically everyone is using organizations' own signup systems. Kide.app is also very popular, especially when there is a need to sell tickets in advance. Google Forms was mentioned by about a third of all respondents, without much correlation with organization.

[PurkkaKoodari]

Issues with signups

This section surveyed opinions on issues affecting signups.

Of people who gave a yes/no answer, about 46% considered the "F5 lottery" a good way to distribute signup slots. About 54% thought it wasn't. For Tietokilta, only 31% were in favor of the lottery, with 69% against.

Over half of all respondents, and 76% of those with a yes/no answer, considered bots a significant issue for popular events. For Tietokilta, the numbers were 57%/43%, with significantly more indecisive answers (36%).

Scalping was considered a significant issue by 35% of all respondents, while 28% didn't consider it to be one. Notably, 36% couldn't answer yes/no. For Tietokilta, over half of the participants didn't give an answer.

[PurkkaKoodari]

Potential solutions

Email confirmation

Description in survey: One solution to promote equality in signups could be an email confirmation, where signing up for popular events would require confirming one's aalto.fi email address before signup. Then, one person could only reserve one spot. The email address used for confirmation would not necessarily be the same as the one used in the signup, meaning one could still reserve your spot for someone else.

The email confirmation could also be necessary for e.g. only the first 5-10 seconds of the signup, such that essentially all interested and active people could get an equal opportunity to sign up.

A clear majority of 69% thought that email confirmation (#57) would be helpful in mitigating signup issues. For Tietokilta, this number was 73%, and for others it was 61%.

Notes from the free text comments:

• 9 people noted that this needs extra consideration if non-Aalto students would participate. This would indeed require configurability if it was to be applied to cross-organizational events.
• 4 people opposed the idea explicitly, especially confirmation being too much hassle.
• 3 people mentioned that it would be nice to be able to reserve multiple tickets for friends. This would not be an issue if the requirement only applies for the first moments of a signup.
• 2 people noted that email confirmation (by itself) would indeed still feature a F5 lottery of sorts.
• Some suggested alternative verification methods, such as student IDs or Aalto/Haka SAML login. Student IDs are likely impossible to implement, SAML might be possible.
• GDPR must be considered when collecting emails.
• Similar methods (requiring pre-login) are already used by some organizations.

Entry randomization

Description in survey: Another solution would be entry randomization, where signups to quickly filled events would be shuffled a certain time (e.g. 5-10 seconds) after the signup opens. This way, getting a spot would require active participation and luck, but the probability would not be affected by internet speed, the device (or multiple devices) used, or bots. In practice, randomization would require email confirmation beforehand, so that the same person can't reserve many "lottery tickets".

Entry randomization (#19) split the crowd, with 39% in favor and 40% against. Interestingly, for Tietokilta, 55% favored randomization while only 24% opposed it; for other organizations, only 32% were in favor with 48% against.

Notes:

• 6 people opposed the idea explicitly, calling lotteries frustrating.
• 3 people believed that this would lead to more cases where groups of friends would be partially left out.
• 2 people noted that the randomization period should potentially take into account the response times of the system. I hope to optimize the system enough that this shouldn't be a significant concern.
• 2 people wished for even longer randomization periods, to allow busy people to have a chance.
• On the other hand, 2 people mentioned that the short period is an essential part of this idea.
• Some considered the current F5 lottery to be fair already.

Many of the above concerns also apply directly to F5 lotteries in popular events. Especially the concern about friend groups was interesting – I don't see how it's much more guaranteed to get your whole group in through a F5 lottery (or if it is, how fair others would consider that).

It's very clear from the feedback that randomization is not necessary for most events, only ones expected to fill up in the first few seconds (or as fast as the signup system can serve requests).

Personally, given the positive feedback from Tietokilta, I think this is worth giving a shot – this fork is built for our website, after all, and the feature is optional.

Online queueing

Description in survey: A third solution would be online queueing, where users could be "in queue" before signup for the event opens, and signups would be accepted in the order of arrival to the queue. Staying in the queue would require active presence, e.g. by filling a captcha regularly. After the signup opens, the rest of the spots would be filled in order of arrival, as usual.

Online queueing (#18) got a clearly negative response across the board, with over half of the responses against it. Interestingly, Tietokilta members were more against this idea (64% answered no), while only 48% of others opposed it.

Notes:

• 10 people directly opposed the idea due to it just being time-consuming.
• 6 people noted that this would mostly move the F5 lottery earlier, especially if the queueing only starts close to the signup opening.
• 5 people mentioned that this could even exacerbate issues with internet connections, as people could be dropped from the queue.
• One person noted that this could be cheated by the same person keeping multiple users in the queue. I think this by itself might invalidate the idea.
• On the other hand, 4 people explicitly called this a good idea and suggested ways to make it better, like an attached Telegram group for hype in the queue.

Online queueing, like its physical counterpart, is designed to favor active and devoted participation. It could even be considered to be the opposite approach to randomization, which gives an equal opportunity for everyone. I think this is very much a value judgment that we, and event organizers, will have to make.

[PurkkaKoodari]

Interest

Only 3% of respondents were not interested to use the results of this project. 92% of the respondents that think they could affect their organization's choice in this matter were interested. This is very encouraging for the project.

Wishes and comments

New feature suggestions

• Pseudonyms/non-public names (Add option to hide name from signups #58)
• Localization (Add language support #17 and other issues in the future)
• Website integration (Create integrable site components #16)
• Per-quota questions
• Options for viewing old events
• Email/Telegram reminders of upcoming signups
• Priorization of people that have attended fewer events
• Webshop-like features (price calculation etc.)
• More captcha ideas

Others

• 2 people wished for stability and load tolerance. This is definitely a design goal for this project.
• 2 people mentioned that easy installation and maintenance is very important. Another design goal.
• 10 answers mentioned features already found in Athene's system: separate reserving and form filling (6x), quotas (2x), overflowing to the open quota (1x), confirmation email (1x).
• 6 people commented on the activity vs. opportunity question I alluded to earlier, with a couple clear opinions each way and 1 wishing that organizers had both choices.
• 1 person hoped for a single Otaniemi-wide system. Somewhat unfortunately, this project overlaps with Prodeko's similar one.
• 1 person thought that nobody should develop a signup system and everyone should just use Kide. Another person thought that Kide.app should just die.

[PurkkaKoodari]

Final thoughts

Huge thanks to everyone that responded to the survey! This will definitely be useful in shaping this project.

The conversation is now unlocked, in case someone wants to discuss.