Hey everyone!

As I promised in the thread in which I asked for your help, now I share the main results of the research of my dissertation. Thanks to the result of the poll, I will share both results related to Steam and Steam users and to engagement itself.

Note: It is going to be a REALLY LONG post which is structured as follows. Firstly, I give you a brief overall introduction of engagement in marketing context (definition and dimensionality). After that, I introduce some information about the sample, my research hypotheses and the testing of them and some other results of my study.
Also if there are any misspelling or something, that's because of the length of this text (and the fact that English is not my native language) :)

Please bump this thread so others can see the results, too.

A brief introduction of engagement

The emergence of the term ’engagement’ in the marketing literature can be dated back to the early 2000s. The concept of engagement itself has its roots in the field of psychology and organisational behaviour. Academics do not have a consensus about the definition and the dimensionality of engagement nor about its actual name (you can come across with terms like ’customer engagement’, ’consumer engagement’, ’brand community engagement’ and so on). However, I agree with those researchers who conceptualize engagement (regardless of its prefix) as a person’s state of mind related to a brand/organisation/community (i.e. the object of the engagement) which has multiple dimensions (namely cognitive, emotional, behavioural and social) and which is a context dependent phenomenon that can have different levels over time.
However, I assumed there could be another dimension, namely ’personal importance’ which would discribe the degree to which a person is engaged to something in terms of missing it from his/her life and the affect of the object of engagement has on the person’s view of himself/herself.

Engagement has a distinctive role in describing the relationship between a customer/consumer (I will use these two denominations as interchangeble though they do not mean the same) and the object of engagement. It means engagement have antecedents (e.g. commitment, involvement, participation, satisfaction) and consequences (e.g. loyalty, trust, word-of-mouth), and engagement can also ect as antecedents of some other concepts such as participation and commitment. There is no mutual agreement about the model of engagement, but according to the literature review I made, this approach can be made (see further the attached image).

My hypotheses

My main research goals after the literature review were to investigate whether Steam has a leading role in Steam users’ game purchasing habit compared to the main online competitors as far as the sources of game purchases are concerned. My other goal was to test whether engagement has five dimensions as I assumed or not. The last objective of my research was to identify if there is a relationship between the importance of given Steam features and the levels of the given engagement dimensions.
So my hypotheses were the following:

  1. H1: Steam has a leading role in Steam users’ game purchasing habit compared to the main online competitors as far as the sources of game purchases are concerned.
  2. H2: Engagement can be described as a five-dimension phenomenon, it has cognitive, emotional, behavioural, social and personal importance dimensions.
  3. H3a: Those users who would miss profile backgrounds to a greater extent has higher cognitive engagement.
    H3b: Those users who would miss Steam Workshop to a greater extent has higher behavioural and social engagement.
    H3c: Those users who would miss Steam Greenlight to a greater extent has higher social engagement.
    H3d: Those users who would miss trading to a greater extent has higher behavioural, emotional and social engagement.
    H3e: Those users who would miss the Community Market to a greater extent has higher behavioural, emotional and social engagement.
    H3f: Those users who would miss badges to a greater extent has higher behavioural engagement.
    H3g: Those users who would miss game reviews to a greater extent has higher social engagement.
    H3h: Those users who would miss community guides to a greater extent has higher social engagement.
    H3i: Those users who would miss Curators to a greater extent has higher behavioural and social engagement.

About the sample

I chose quantitative approach to test my hypotheses. I constructed an online survey which you completed. Eventually I gathered 980 answers out of which 956 answers were usable (I had to exclude the obvious trolls and those who showed high inconsistency among the answers). So my usable sample was 956 (n=956).
Note that the following statistics refer to the sample and can not be generalized as the sample only included members of the SteamGifts community.
A brief description of the sample:

  • Gender-related distribution: 87% male, 13% female
  • Age (years): a mean of 25,7, a minimum of 13, a maximum of 65, a mode of 25 and a median of 24,5.
  • Education: 2% has elementary school or less, 38% has grammar school/high school or equivalent, 60% has university degree
  • Occupation: 44% student, 30% employed, 14% unemployed, 10% self-employed, 1% homemaker, 1% other.

Further information about Steam users:

  • Over 91% of you sign in the Steam client almost everyday or more than once per day (while only about 60% of you visit the Steam webpage and play games on Steam that frequently)
  • Approx. 90% of you are satisfied with your overall experience with Steam (it’s the cumulative percentage of respondents who answered with 5, 6 or 7 to the question ’ How would you rate your overall experience/satisfaction with Steam on a scale from 1 to 7?’) – Gaben must be proud of it :D
  • The average Steam account is 4,53 years old (with a mode of 3 and a median of 4), the average user has 550 games and 77 friends on Steam, the average profile level is 28 (with a mode of 20 and a median of 22), and finally the average user has 46 badges (with a medain of 20)

I tested whether the sample is representative based on the hardware and OS statistics. I compared the data with the Steam hardware and software survey. Apart from the number of CPU cores, the sample can be deemed to be representative of the Steam users (for these variables: manufacturer of the CPU, manufacturer of the GPU, OS).

Test of the hypotheses

Test of H1
To test whether the H1 hypothesis is correct, I compared the frequencies of the usage of the different sources you use for buying games. There were 3 questions in the survey related to this topic: where have you ever bought a game (’ever’), where do you buy games most frequently (’frequently’), where would you buy a game if it had the same price at all sources (’preferred’).

I compared the frequencies of the different sources with which they were chosen at each questions. The Top 3 sources are the following:

  1. Steam: 98,4% ever, 88,1% frequently, 77,6% preferred
  2. Humble Bundle: 79,9% ever, 57,1% frequently, 13,1% preferred
  3. Bundle Stars: 56,8% ever, 22,8% frequently, 0,6% preferred
    (Note: GOG was the third in the ’preferred’ comparison with 5,2%).
    According to these results, my H1 hypthesis was correct.

Test of H2
To test the H2 hypothesis I used factor analysis. After checking that the subsets of items are reliable and that they can be used for factor analysis, I investigated the optimal number of factors. Supporting my hypothesis, I got that 5 factors can be deemed as ideal (based on a number of criteria), so I investigated whether the a priori assumption of dimensionality was correct. I looked at the rotated component matrix (I used Varimax rotation), and the results showed my assumption was correct as the subsets of each dimensions loaded on the same component (for example the items I used to measure emotional engagement all loaded on Component number 1). Only a few items loaded significantly on another factor than the majority of the subset items, mainly items from the assumed personal importance subset (2 of them loaded approx. to the same extent on the component representing the cognitive dimension). Thus I run another factor analysis with 4 factors and as I thought, the former personal importance and cognitive factor merged into a new one, while the other factors remained the same. However, because only a few items showed significant correlation with another factor, I accepted my H2 hypothesis with the note that maybe a 4 dimensional approach can be correct as well.

Test of H3
At first I looked at the means and the medians of the questions ’On a scale from 1 to 7, how much would you miss the following features if they were removed from Steam?’ (see further the attached image).
After that I created three groups based on the answers to the above mentioned questions:

  • Group 1: Those who would not miss the given feature (answers 1, 2, 3)
  • Group 2: Those who are neutral about the removal of the feature (answer 4)
  • Group 3: Those who would miss the given feature (answers 5, 6, 7)

After this I used ANOVA (analysis of variances) to determine whether there is a relationship between group membership and the level of the given dimension (for this I used the factors from H2).
The results:

  • My H3c, H3e, H3g, H3h and H3i hypotheses were correct and with the exemption of the relationship between the importance of the Curators and the behavioural engagement, all relationship was positive (meaning that the more a user would miss the given feature the higher his/her respective engagement is).
  • My H3b and H3d hypotheses were partially correct, in both cases the relationship between the importance of the feature and the behavioural was not significant.
  • My H3a and H3f hypotheses were not correct.

Further notes

I would like to thank you once again for your contribution to my research :) If you have any questions regarding it, feel free to ask it in the comment section below. I try to answer all of them.

View attached image.
View attached image.
8 years ago*

Comment has been collapsed.

Great work, but you mention yourself that the sample is biased as you only sampled SG users (forum users at that). Do you plan to sample users from other sites as well? If not, this would be something any thesis advisor should ask you to do.
GL with your dissertation.

Edit: I doubt your sample is representative. You mention, you compared your sample to Steam's hardware survey. Of course SG users sample Steam users, as hardware is mostly based on the available money, but why should it hold true the other way? I would think that SG forum users are more active in general than the average Steam user.

8 years ago*
Permalink

Comment has been collapsed.

I'm not too familiar with all the terms the OP use but I'm doubting the sample to be representative as well. If you look at the education you see that 60 % have an university degree. It is unrealistic to believe that this is represantitive for the whole Steam Community. It rather shows the willingness of people familiar with scientific work to help you. The work may still have it merits but I would be prudent on how to formulate its representativeness to others.
It would be interesting to gather information from other sources with another level of education and look how well it compares. Looking at the timeframe this may be in the works or come for the future. After all the survey is from last month and a dissertation takes normally a bit longer.

8 years ago
Permalink

Comment has been collapsed.

As I mentioned in the description, the sample is only representative regarding manufacturer of the CPU, manufacturer of the GPU and OS and not regarding other variables such as education or age.

8 years ago
Permalink

Comment has been collapsed.

Yeah, the best way would have been to post the survey on Steam forums but according to the Steam Online Conduct they can not be used in connection with surveys.

The sample is only representative for the mentioned variables, you are right, SG users might have higher (or at least different) engagement toward Steam than the average Steam user.

8 years ago
Permalink

Comment has been collapsed.

Have you tried to contact Steam and ask them if they would support your research, by allowing a survey (or best case feature it)?

8 years ago
Permalink

Comment has been collapsed.

No I haven't (actually I wanted to use SG in the first place baceuse it was easier to motivate participation here - another possible bias).

8 years ago
Permalink

Comment has been collapsed.

Other thought: Have you asked cg, if he could provide you with numbers of users visiting your old thread in the given timeframe, so you can determine the percentage of people participated in the poll? (If those numbers exist)

8 years ago
Permalink

Comment has been collapsed.

You mean to calculate a kind of 'response rate'? Yeah, that could have worked but I did not think it was necessary.

8 years ago
Permalink

Comment has been collapsed.

"Over 91% of you sign in the Steam client almost everyday or more than once per day"... well well well, you found out my dirty little secret! xD

On another note, try to be a little less black & white with your statistics (i.e. dont entirely discount or accept any hypothesis).
I know it sounds pedantic, but the term 'correct' implies quite a strict output from a stats test. I try to stick with terms that factor in that stats are not conclusive, like "statistically significant" etc., and include how close or far you are from you critical value (p).
Also, what were your stats parameters? i.e. one or two-way ANOVA, confidence level, and choice of post hoc.

When is your write-up due? All the best finishing it off Szat :)

8 years ago
Permalink

Comment has been collapsed.

Yeah, you are right, I could have used phrases like 'supported' or 'not supported' instead of 'correct' and 'not correct' here but I wanted to as simple as possible because I didn't know how many of you have relevant statistical knowledge.

I used one-way ANOVA, with a confidence level of 95%. Post hoc tests: Scheffe and Tukey (these were recommended by my advisor).

8 years ago
Permalink

Comment has been collapsed.

Bump for nice statistics.
I didn't dig how you tested the second hypothesis, so my next question might be rooted in that misunderstanding.

How did you distribute the dimensions of engagement among the Steam features? Could it be that the incorrect hypothesis formulation could have led to it's acceptance our rejection?

8 years ago
Permalink

Comment has been collapsed.

On a side note: could there be anothr source of bias within the SG community as only the active forum participants (who might have higher levels of engagement) might be present in your sample?
I have to admit I'm missing out on forums lately and I totally missed the point where I could take part in your survey, sorry.

8 years ago
Permalink

Comment has been collapsed.

Do you mean the assumption upon which I selected the dimensions of engagement? If so, I used the literature review (e.g. in the case of badges) and my own personal assumptions regarding the possible relationship between the Steam features and the levels of engagement. I did not mention here (but in my dissertion I do) that there can be other features to be investigated as well as other possible dimensions of engagement to be investigated (e.g. Steam Workshop - emotional engagement).

SG forum users might have higher levels of engagement, but it is not clear whether towards SG or Steam in general. But it is worth noting, for sure :)

8 years ago
Permalink

Comment has been collapsed.

If levels of engagement and dimensions of engagement are the same, then yes, it is what I had in mind.
The presence of personal assumptions might be a minor flaw: what if some of your hypotheses were rejected because your personal assumption was incorrect? Say, your hypothesis 3a says tests for connection between profile backgrounds and cognitive engagement. Why do you only want to test it against cognitive engagement? What are the reasons for that? Could you test it against social engagement? Againt a combination of emotional and behavioural engagements?
I guess, you can actually try finding out the connection between features and dimensions by testing a feature against different (all of them separately and possibly double/triple combinations too) dimensions - seems like an interesting research point (i.e., what actually drives people to use those features)?

8 years ago
Permalink

Comment has been collapsed.

Levels of engagement refers to low and high (and everything in between) and dimensions of engagement refers to cognitive, emotional, etc.
Results of hypotheses are not rejected based on personal assumptions but based on the data. I mean, hypotheses are of course based on the researcher's assumption of what to investigate (and what connections might be among the given variables). But you are right, there can be other possible relationships I did not investigate in this study.

8 years ago
Permalink

Comment has been collapsed.

I understand that the rejection is based on a data. What I mean is that if the hypothesis is false because it is based in a wrong assumption it will be most defitely rejected based on the data because it was intrinsically false, so, yes I think it will be beneficial to see more hypotheses tested and a more systematic approach to hypothesis formulation.
By the way, what is that reference you mentioned above about the connection between badges and behavioural engagement?

8 years ago
Permalink

Comment has been collapsed.

Yeah, but this is hypotheses are for; to test whether an assumption is correct or not :)
I don't know if you have access to ScienceDirect or not but I used this and this articles as a basis in connection with badges and their impact on behaviors.

8 years ago
Permalink

Comment has been collapsed.

I'm most surprised by the 2% having elementary school education or less.

8 years ago
Permalink

Comment has been collapsed.

I am really surprised that there are only 13% females on SG. Any way, thanks for sharing!

8 years ago*
Permalink

Comment has been collapsed.

Deleted

This comment was deleted 8 years ago.

8 years ago
Permalink

Comment has been collapsed.

By this community I meant SG

8 years ago
Permalink

Comment has been collapsed.

Bump.

8 years ago
Permalink

Comment has been collapsed.

Bump.

8 years ago
Permalink

Comment has been collapsed.

Sign in through Steam to add a comment.