First Release

On Friday 8 April, we launched the Data Miner app on facebook.
The app can be found on http://apps.facebook.com/datamineralpha/.

Only the core feature was available at that time.
Users could mine using the mouse or keyboard and could read articles once mined.
Till now these articles were selected from a predefined set of articles.

Also included in the application was a link to a google form.
This form can be found on link.
It is based on the CSUQ test.

With the questionnaire, we hoped to find out how much users like the application and if it is interesting enough.
Of course the app isn't that interesting right now, because we use dummy articles.
Also the reward system isn't there yet, so it is quite difficult to attract people day after day.
After a full week and active pushing friends and family to the app, we got 10 responses on the questionnaire.

We conclude the following after this first iteration:
- The users weren't pleased with the graphics.
- The icons were to small.
- Users with Safari can see through the article making it very difficult to read.
- Users asked for a legend for the different materials.

+ Most users are medium satisfied with the application and hope to see more soon.
+ Most users find it simple to use the app.
- Because there aren't that many different social news games, it was difficult for most users to compare it with something else.
Also they didn't know if it had all the functionalities they expect it to have, cause they didn't expect that much.
- We score neutral on the effectiveness of our app. This because we still use dummy articles. The test users know what we want to do and think this can be effective, but right now they rather watch the monkey out of the tree. (English with hair on it)
+ Except for the Safari 5 tester, all other testers felt comfortable with the app.
+/- The opinions about the concept are divided.

We also noticed that most users reach their articles within an acceptable amount of clicks or keystrokes. Some users also try to dig as much ground as possible without getting to an article. This is not really the point of the application, but as long as they like doing it, it's fine for us. We can't ban them for doing it. We will have to find some way of removing these users from the usage statistics, as they raise the average amount of clicks/buttons pressed by a large amount.

That's it for right now.
During the second iteration we will:
+ Update the graphics for a more pleasant experience.
+ We will add the reward system to lure users back to the app.
+ We will widen our test group.

On hold:
- The Safari problem.
- The legenda.

Till next time!

Team AnarCHI

Planning & first iteration

First iteration/release

We decided to skip the tests we planned for previous weekend and decided to directly test on our digital prototype. The tasks our users have to perform changed, but the questionnaire stays the same.

What will we test?

During our first release we will test our core-functionality: the mining and rating of articles. We would like to know if users:

1. find it easy to navigate.

2. enjoy mining and rating articles. (do they like the idea? Would they prefer it differently?)

3. actually want to rate articles.

How will we test this?

We will test the core-functionality of our application in two ways.

We will first conduct an offline test using a dummy flash application (the weekend of april 9th). The offline test will allow us to present test-users with a questionnaire. (We will use a slightly modified version of the given questionnaires, more on that in a next post). The questionnaire will help us determine whether or not the users actually enjoyed our application and of the general usability of our application.

The second part of the test will come as a first release on facebook (the week of 17/04).

During both parts we will keep track of the number of mouseclicks and keystrokes the user uses. We expect about 25 keystrokes between mining articles and an average of 1.1 mouseclicks between mining articles (since mining an article simply involves clicking it). Rating articles involves an 'interesting', 'meh' and 'boring' button in addition to a close button. We expect each of these buttons to be pressed roughly the same amount of times. Should the close button be used more than we hoped, this could indicate that the user isn't interested in rating articles or he/she doesn't understand it.

Who will we test on?

The offline part of the test will be conducted on 6 people in total. These people will be chosen at random from our own social circles. The second part of our test will be on facebook. We will post the application on our facebook wall.

Planning

09/04:

Working flash application consisting of dummy-sprites. It should be possible to move the miner and to mine/rate articles. Rating an articles will have no actual impact whatsoever.

We will test this bareskin flash application the weekend of april 9th. This will happen offline with people from our own environment (6 in total).

17/04:

First 'real' release on facebook using the bareskin flash application developped earlier. We will review the feedback received during the weekend of april 9th before the release, as well as add some better looking sprites.

24/04

Between april 17th and april 24th we will let the application run its course on facebook. We will gather feedback (using the method described above) and will use this time to develop the non-core-functionality (invite a friend, statistics, etc.).

Week of 25/04:

During the course of this week we will gather and interpret the feedback from our first 'real' release on facebook. We will improve our core-functionality based on this feedback and add the extra functionality developped between april 17th and april 24th. We will then release the application a second time on facebook.

Week of 02/05:

We will let our second release run its course on facebook and gather feedback.

Week van 09/05:

We will review the second release, improve, and release a third and final time.

Week of 16/05:

We will use this week to work on our final report as well as interpret the feedback from our third and final release.

Further testing

For the next evaluation of our application, we will use the paper prototype for the last time. We will now describe how this testing will happen.


What will we test?
We have 3 core functionalities in our application: mining and rating articles, adding friends and getting statistics of a user compared to his friends. This is why we made 3 special tasks to test the usability of these features.
We also wonder if the detailed statistics can be found by the users. The screen is not accessible in a single click, so we added a fourth task to test if our users can find it.
In addition to these detailed information, we also want to know the overall feeling our users have of the application.

How will we test it?
The users will perform these tasks in the order below. They get the next task as soon as the previous one is finished.
We will measure both the time it takes to perform a task, as the amount of clicks needed to perform it.

1. add a friend
2. compare your statistics to those of your friend "Drill Bill"
3. mine and rate an article
4. find the detailed statistics of the category "sport".

Once the users are finished with the prototype, we will ask them to fill in an additional questionnaire. This should help us measuring the overall usability of the application and the feeling users have of the application. We want to keep the amount of questions lower than 20. Otherwise our users would get bored with the questions and might give the application a worse rating, just because they are getting bored.
We have chosen to use the CSUQ test. This seems the best one to represent what we want to know. Other tests are more focussed on the job of a person or the errors the application gives. We also don't want a too short question list, as that doesn't provide enough information. The 19 questions of CSUQ seemed like the best way to go.

Who will we test on?
As our application is designed for adolescents, we will try to test this on adolescents. This means people between 14 and 22 years old, both boys and girls. It is best if these people don't know too much of informatics. Everyone in our group will test 2 people, so we have 8 testpersons in total. For the ease of finding people, we will ask people in our direct surroudings during the weekend. In Leuven we don't have access to many people in our testing range.

Report 1

The following link contains our first report for the course of human-computer interaction. In this report we explain what we did during the first step of our project. We go into detail about our concept with the aid of a storyboard and a screen-transition diagram. Then we continue with explaining our paper prototype and what conclusions we took from these first tests of our application. Finally we also make some conclusions about this iteration and plan on what we will do in the next iteration.

We wrote the report in English, so our non-dutch-speaking readers can also read the report.

http://www.mediafire.com/?cb3824a2ffh1bww/

Paper prototype - 3rd version

We tested this third prototype with housemates who are not computer scientists. This way we could get some feedback from people who have a different view on computer programs.
The feedback we got from this was generally positive. Although some testers still had their doubts about the concept, the interface seemed to be doing its job very nicely. These are the relevant remarks:

• The text in the indroduction screen is not clear enough.


• The button to indicate which friends are taken into account for the value of articles - a green/red “stoplight” - is confusing. People don’t know what it’s for until it has been explained explicitely.


• The button to rate an article neutral was unclear.


• Using the terms Like/Dislike for rating an article causes wrong usage: interesting articles about bad events were rated negative although for our system they should be rated positive.


• When looking at the statistics, some people wanted more specific information, such as a further refinement of the categories.


• One of the test persons thought the button to close a window would close the entire application, and didn’t manage to return to the mining game.


• We had a facebook like-button to the starting screen of the application. However, this was only reachable when starting up the application while it should be accessible anywhere.

Because there were still quite a number of (small) remarks, we are planning to do one more paper prototype iteration. We will make the following changes:

• The “stoplights” in the friends list will be replaced with a checkbox. The color of the box of the friend will also help indicate whether or not they are included in the calculations.


• The rating buttons will have a visible name in addition to the icons, clarifying the purpose.


• The introduction message will be slightly more elaborate to clearly specify the goal of the game.


• It will be possible to view the statistics of categories in general, as well as more specific stats within a category.


• The facebook like-button will be visible everywhere, although we haven’t figured out exactly where on the screen it will be.

We decided not to change the close button, since we believe that when the application appears in the context of a webbrowser it will be clear that it doesn’t close the entire application.

Paper prototype

Since we had to re-do a lot of the work of the previous sessions, we are running a little behind on schedule. We started with paper-prototype tests of our application on Wednesday. We started with a prototype to test on our own. Not all popups and mouse-over effects were there, as it was just to see if we didn't forget some major functions. In the picture below, you can see how the prototype looked like. We already kept in mind some of the things we learned from the paper prototype tests of our previous concept. Users want a close-button for the menu items, and don't want to click the menu on the left. They are lost if we just give them a menu, they want close buttons. We decided having both the menus and close buttons available would be best.

Once we had a good feeling about it, we made a second prototype. This time we tested it on people sitting in the hall of the computer-science department. We noticed people didn't know where the mining button was for. They kept clicking on it, while they were already on that screen. This button remained there from the time we didn't have the close buttons. In the next prototype we will remove this button.

The users also wanted to be able to select which friends are being used to determine what articles they are mining. To achieve this, we will add an extra button in the friends list. By clicking this, a friend can be enabled or disabled.

We are now testing our new version of the paper prototype with new users and hope the interface will be better with our new improvements.

Working out our new concept

We decided to work out the idea for mining and rating news. We will first explain how this new concept will work, and then explain what we think the benefits are compared to our old concept. However, if users prefer the old concept we will go back to the previous one.

A user will have the possibility to mine freely in the underground. In this underground, he will find different minerals that represent articles. When his friends haven't rated the category of the article or the total ratings aren't positive or negative, it will be a rock. A more valuable mineral will represent an article category his friends liked. The more they liked it, the more valuable mineral will represent it. A diamond will represent the most valuable article categories. A non-valuable icon, like talc will represent an article category that was rated negatively by his friends.

The user can earn badges with the actions he does in the game. There will be badges for reading articles attached to a specific mineral. For example "diamond lover" if the user read 100 diamond articles or "persevering" if he read a lot of articles rated as talc. The user is still allowed to choose like or dislike after reading the article without consequences for the badges. The rating system is to help his friends, so the reward for pressing the right button is that the user will get more relevant news later on, depending on his actions. We will only attach badges to how often the user uses the application, not how he uses it.

Since we have a clear view of what categories of articles a group of friends like, we can create graphs to indicate the common interests of the friend group. This way a new user can find out what his friends like, just by adding them. Without even reading a single article. The graph could also show the difference between what the user likes and what his friends like, allowing him to remove friends that have totally different interests. If you just want the articles you like, just don't invite other friends.

For finding friends easily and making it easy for our application to spread, we would make this as a facebook application. This way users don't have to register and they can just select their friends in a list to play together. It is also possible to give users the option to post badges on their wall when they earned them, allowing to attract more new players in our application.

So what are the advantages of this new concept?

• The users have a clear motivation for playing the game. If they rate articles, they will get news focussed on what they like. If his friends rate a specific category, this will also give the current user more news from that category.
• The application still has badges, but now the badges also make it possible to attract new players. In the old application badges only made the current users attached to the application.
• We think the application should be simpler and the concept more clear. We dropped the highscores and the selling system, keeping the application focussed on 1 thing.
• Users can now indicate if they didn't like a specific article. This will make sure the current category will be shown less often. They won't feel we are bullying them with articles they don't like, we are just trying to learn what they like.

We will create a storyboard for this new concept in the following week and start to test it with a paper version once that is finished.