I have launched or participated in more than 40 educational projects and products
A new way to measure overconfidence bias, published paper, 2013
Kolesnikova, D., & Belianin, A. (2013). Prognosis Calibration in Binary Choice Tasks. Psychology. Journal of Higher School of Economics, 10(4), 42-66.
This works focuses on the problem of confidence in decisions made. Existing studies of prognosis calibration (defined as the ratio between the correct response frequency and the subjective evaluation of response correctness) have revealed that people tend to overestimate the probability of correct responses ("overconfidence effect"). The authors hypothesized that subjective estimates of confidence in decisions made and the accuracy of prognosis calibration depend on the method used to measure confidence.

An experimental study was performed in a sample of school students (N=50) and university students (N=36). The participants answered common knowledge questions and evaluated the degree of confidence in the correctness of their answers on verbal scales, using either a standard scale or a duplex scale (rating their confidence in both chosen and rejected answer options). Finally, the participants made bets that the chosen alternatives would be correct.

The results indicated that the "overconfidence effect" was weaker in the duplex scale condition, confirming the hypothesis. We explain this finding by the notion of residual uncertainty that remains when two answer options are compared, but not in the standard scale setting, when the respondents' attention is concentrated only on the chosen answer option. The association between the level of confidence and prognosis quality had an inverse linear shape in the standard scale setting and a parabolic shape in a duplex scale setting. Finally, we found that in cases when the decisions were rational and the confidence ratings were consistent, bets predicted prognosis quality more reliably than verbal estimates did.

This work started at Teachers College, Columbia University, New York in 2012 as research for my Master thesis and was finished in collaboration with Prof. Alexis Belianin at National Research University Higher School of Economics, Moscow in 2013.
Comparison of ipsative and Likert-type scales
Term paper 'The Theoretical Framework for Construction and Comparison of Interest Surveys based on Normative and Ipsative Scales'
2019 – present
The work on this paper consisted of the following stages
  1. Based on Social-Cognitive Career Theory the framework for vocational interests questionnaire in Data Science was constructed. Popular occupations in Data Science were decomposed into 4 dimensions: math, programming, analytical thinking, and leadership. Potential career choice was modelled as a combination of self-efficacy, outcome expectations, and occupational interests.
  2. 44 statements were developed based on the framework. These statements were prepared for both Likert-type and ipsative questionnaire.
  3. Based on three cognitive laboratories, 40 statements were chosen for the next stage, numerous statements were re-written.
  4. Questionnaires were piloted. 84 respondents participated in the pilot study. Classical Test Theory was used to assess the quality of the items formulated for Likert-type questionnaire. Based on this analysis, statements were re-written once again.
  5. New versions of the questionnaires were tested in 5 cognitive laboratories. They were re-written.
  6. Randomised design was developed for the next round of surveys exposure. The last step will be to gather new dataset for both questionnaires. The analysis will include IRT for item quality assessment for the Likert-type survey and transformed data for the ipsative survey. Then SEM will be used to test whether the data fits the theoretical framework. Finally, prediction models will be built to test whether the questionnaire could be useful for prediction of one's vocational interests.
Maxim Skryabin
Denis Federyakin

Further details
This work has been developed during my Master's program in Educational Measurement at National Research University Higher School of Economics, Moscow
Assessment of student progress at the online school, pilot research
Pilot research on the knowledge progress at SkillFactory, for students at Data Science and Analytics programs, 2020
It was necessary for the online school to understand whether its students develop during their studies. For the pilot study two courses were chosen: 12-month program on Data Science and 6-month program on Analytics. The work on this project consisted of the following steps:
  1. Pedagogical specifications for both courses were revised
  2. Based on the specifications, two knowledge tests were designed in collaboration with domain experts (15 items in the Analytics test, 25 items in the Data Science test)
  3. 47 and 255 respondents participated (Analytics and Data Science correspondingly), they represented 16 and 29 cohorts who had started studying at different time
  4. CTT and IRT were used for the analysis of Analytics and Data Science tests respectively
  5. It was shown that, at the Data Science program, students who started to learn earlier showed significantly higher knowledge level than more recent cohorts. This difference was not so significant for the students of the Analytics course. Some knowledge gaps were discovered. The tests were revised based on the item quality analysis

I initiated this research while working in SkillFactory as a product manager at Quality Lab. The goal of our team was to raise the quality of our educational content
Game-based assessment of 21st century skills
Four online pilot games created in two weeks, Feb 2018
The idea was to create games for 21st century skills assessment. All games were based on the famous experiments from behavioral economics or cognitive science. As a result 4 online games were created from scratch just in two weeks with a team of 4 people. Games were piloted with a 200+ children in cooperation with an educational platform InternetUrok.

Four competencies that games were aimed to measure:
– Ability to learn (the game was based on the theory of concept formation; Voiklis, J., & Corter, J. E. (2012). Conventional wisdom: Negotiating conventions of reference enhances category learning. Cognitive Science, 36(4), 607-634)
– Resource management (the game was based on the ability of respondents to act rationally; Shin, J., & Ariely, D. (2004). Keeping Doors Open: The Effect of Unavailability on Incentives to Keep Options Viable. Management Science, 50(5), 575-586)
– Ability to concentrate (the game was based on the classical n-back task; https://en.wikipedia.org/wiki/N-back)
– Ability to manage risks (this game was more about personal preferences than abilities; it was based on the Tanaka, T., Camerer, C. F., & Nguyen, Q. (2010). Risk and time preferences: Linking experimental and household survey data from Vietnam. American economic review, 100(1), 557-71)

This project doesn't have any scientific value, as I understand right now. However, it is because of this initiative that I decided to dive deep into psychometrics.

Further details
Try a prototype:
Login: testphd
Password: passphd20

Experimental research on public goods
Research assistantship at the Lab for Experimental and Behavioral Economics, HSE; 2010, 2012-2013
I participated in data collection for public goods experiments in Perm, Tomsk & Moscow. In particular we gathered data to explore the phenomenon of spite. The spite could arise when respondents saw that someone contributed more points to the public good than they did. This effect was found to be especially strong in Middle East, Russia and Eastern Europe (Herrmann, Gachter, Thoni, 2009).

The goal of the experiments conducted by our lab was to explore true motives of the people who showed such anti-social behaviour. It was found that such motives are numerous: availability of the option to punish, competitiveness, preemption, etc.

The conclusions were based on 300 paid participants. 128 from Moscow, 76 from Perm, 96 from Tomsk. The experiment was realised in the software z-tree.

Prof. Alexis Belianin
NGO MyStory
I am a founder of the startup called MyStory ('Obraz jizni' in Russian), 2014-2019
The initial goal was to create an opportunity for every child to explore the world around. 3.5 years ago we started with online-meetings for teenagers living in distant orphanages, today we work with both young people from regional orphanages and high-school students from all over Russia and CIS. By working with two groups of participants we are able to be financially stable. Currently our focus is on offline activities, for example city camps.

  • 10 products in career counselling (online meetings, trainings, school courses, camps, etc.)
  • More than 18k participants in online course
  • More than 500 hours of events
  • More than 400 unique participants from all over Russia in summer camps
  • 17 regions including Armenia and Ukraine
  • Won the competition for funding from federal government, 2018; from Moscow government, 201
Python course in SkillFactory
SkillFactory (Data Science School, more than 4,000 students, 30 online-courses). Launched Quality Lab - the department for quality control. As a product-manager increased gross profit from 30% to 55%, quality from 23% to 60% (NPS). Launched 'New products' department (free experimental products).
Economics based on games course
The idea was to show that Economics as a science doesn't study only finance and business but analyses human behaviour under different settings: from shopping to dating. This project is currently on hold, though I conduct some rare events on non-standard economics and teach classes in the school on weekly basis.

2010-2016 results
  • More than 800 participants overall
  • From 7 to 60 years old
  • 20 events in 1 year
  • 2 year-courses for schools are developed and conducted
Teaching at the school
Taught Economics for school children in High School, 2015-2017 School №1533,
Taught Behavioral Economics for school children in Middle School, School Intellectual, 2014
Designed and taught 10+ workshops on Statistics and Economics, 7-12 year old children, Polytechnic Museum, Moscow 2015-2018 (Independant)

Board game development
Developed a board game about communication for top-managers, Masterskaya O. Zamyshlyaeva
Teaching related to career consulting
Workshops, webinars, online and offline courses, 2014 - 2019
  • Developed free online course about MyStory approach to career counselling and published it at Stepik.org, 2019
  • Conducted more than 30 talks and workshops [RUS] for school teachers on the methods of career counselling and support of young children, MyStory, 2014-2019
  • In partnership with a leading online Russian school launched 2 online course for teachers about career counseling that attracted more than 18K participants and certified more than 10k teachers (one course in now archived, another is available here [RUS]), 2017, 2019
  • Developed a course on career choice for 10th grade, 2017-2018, Lyceum of HSE (MyStory

Online course at Stepik.org
For teachers about MyStory approach to career counselling, 2019
Developed free online course about MyStory approach to career counselling and published it at Stepik.org. It includes 6 chapters on how to organise engaging meetings for young people and professionals from various occupational spheres at school.

    • 2021. NRU Higher School of Economics, MA Educational Measurement
    • 2013. Columbia University, Teachers College, MA Cognitive Studies in Education
    • 2011. University of London, International program, BSc Economics, First-Class Honours Degree
      • Sep 2020 – now. R&D Senior Product Manager, uchi.ru (Online Platform for primary school students; more than 8.5 mln users). The task is to launch R&D department.
      • March 2019 – May 2020. Head of Student Experience, SkillFactory (Data Science School, more than 4,000 students, 30 online-courses). Launched Quality Lab - the department for quality control. As a product-manager increased gross profit from 30% to 55%, quality from 23% to 60% (NPS). Launched 'New products' department (free experimental products)
      • 2014 – 2019. Founder, NGO MyStory (21st Century Skills to the Youth), obrazobraz.ru; Launched more than 10 products on career counselling, including online course for teachers (more than 18k registered, 10k finished; camps for young people - more than 400 participants from all over Russia, etc); Won the competition for funding from federal government, 2018; from Moscow government, 2017
      • June – Aug 2020. SkillFactory. Introduced metrics of content quality, student knowledge level, retention.
      • Oct 2019 – now. Developed an interest inventory for IT newcomers. The goal is to compare an ipsative and Likert scales functioning.
      • 2012 – 2016. Taught Economics and Behavioral Economics to kids, high school students, university students, economics teachers; designed more than 50 educational courses and events.
      • 2010 – 2014. Research assistant, International Laboratory for Experimental and Behavioural Economics
      • 2012. Research assistant. Columbia Uni, Teachers College; James Corter's Cognitive Science Lab
      • 2020. HSE Alumni Awards
      • 2018. A member of YPO (Young Presidents Organisation) Impact program
      • 2018. Winner, second prize, Innovate for change, Berlin
      • 2017. Included in the list of innovators in education in Russia
      • 2014. Absolute winner, Competition of Innovations in Education
      • 2010. Letter of Commendation from University of London,
      • 2008, 2009. Potanin Fund scholarship student (excellent marks and teamwork skills)
      • 2009. Oxford Fund scholarship student (excellent marks and activities)
      • Data Analysis in R
      • Statistics (up to panel data analysis)
      • Psychometrics (CTT + IRT)
      • English – IELTS 8, TOEFL 110