Participation at TASS 2012 is closed.

Participation

Participants are required to register for the task(s) in order to obtain the corpus.

Registration must be done by mail to .

Submissions

Experiments

Experiments must be submitted in a plain text file with the following format:

twitit \t polarity \t topic

where twitid is the twit ID for every message in the test corpus, the polarity contains one of the 6 valid tags (P+, P, NEU, N, N+ and NONE), and the same for topic.

Although the polarity level must be classified into those levels and the results will be evaluated for the 5 of them, the evaluation results will include metrics that consider just 3 levels (POSITIVE, NEUTRAL and NEGATIVE).

Participants may submit results for one or both tasks.

Submissions for any of the tasks must be done by email to .

Reports

We invite participants to submit a report describing the experiments and discussing the results.

Papers should follow the usual SEPLN template given in the author guidelines page. Reports can be written in Spanish or English. In this case there is no limitation in extension as they will be included in the electronic working notes of the conference.

Submissions must be done by email to and the deadline is August 19th, 2012.

Results

15 groups registered and finally 9 groups sent their submissions. Results are listed in the tables below.

The detailed results for each run, the relevance judgement files (the correct labels for the test set) and a PHP script for performing your own evaluation can be found in the Downloads section of the Corpus description page.

Task 1. Sentiment Analysis (5 levels + NONE)

Run Id Group Precision
pol-elhuyar-1-5l Elhuyar Fundazioa 65.29%
pol-l2f-1-5l L2F - INESC 63.37%
pol-l2f-3-5l L2F - INESC 63.27%
pol-l2f-2-5l L2F - INESC 62.16%
pol-atrilla-1-5l La Salle - Universitat Ramon Llull 57.01%
pol-sinai-4-5l SINAI - UJAEN 54.68%
pol-uned1-2-5l LSI UNED (tamara & jorge) 53.82%
pol-uned1-1-5l LSI UNED (tamara & jorge) 52.54%
pol-uned2-2-5l LSI UNED 2 (angel, juan manuel & ana) 40.41%
pol-uned2-1-5l LSI UNED 2 (angel, juan manuel & ana) 39.98%
pol-uned2-3-5l LSI UNED 2 (angel, juan manuel & ana) 39.47%
pol-uned2-4-5l LSI UNED 2 (angel, juan manuel & ana) 38.59%
pol-imdea-1-5l IMDEA 36.04%
pol-sinai-2-5l SINAI - UJAEN 35.65%
pol-sinai-1-5l SINAI - UJAEN 35.28%
pol-sinai-3-5l SINAI - UJAEN 34.97%
pol-tudelft-5y6-5l TUDELFT 34.05%
pol-tudelft-3y4-5l TUDELFT 33.46%
pol-tudelft-1y2-5l TUDELFT 33.04%
pol-uma-1-5l UMA 16.73%

Task 1. Sentiment Analysis (3 levels + NONE)

Run Id Group Precision
pol-elhuyar-1-3l Elhuyar Fundazioa 71.12%
pol-l2f-1-3l L2F - INESC 69.05%
pol-l2f-3-3l L2F - INESC 69.04%
pol-l2f-2-3l L2F - INESC 67.63%
pol-atrilla-1-3l La Salle - Universitat Ramon Llull 61.95%
pol-sinai-4-3l SINAI - UJAEN 60.63%
pol-uned1-1-3l LSI UNED (tamara & jorge) 59.03%
pol-uned1-2-3l LSI UNED (tamara & jorge) 58.77%
pol-uned2-1-3l LSI UNED 2 (angel, juan manuel & ana) 50.08%
pol-imdea-1-3l IMDEA 45.95%
pol-uned2-2-3l LSI UNED 2 (angel, juan manuel & ana) 43.61%
pol-tudelft-5y6-3l TUDELFT 43.61%
pol-uned2-4-3l LSI UNED 2 (angel, juan manuel & ana) 41.20%
pol-uned2-3-3l LSI UNED 2 (angel, juan manuel & ana) 40.43%
pol-tudelft-3y4-3l TUDELFT 40.27%
pol-tudelft-1y2-3l TUDELFT 38.45%
pol-uma-1-3l UMA 37.61%
pol-sinai-2-3l SINAI - UJAEN 35.83%
pol-sinai-1-3l SINAI - UJAEN 35.58%
pol-sinai-3-3l SINAI - UJAEN 35.11%

Task 2. Trending topic coverage

Run Id Group Precision
top-l2f-2 L2F - INESC 65.37%
top-l2f-1y3 L2F - INESC 64.92%
top-atrilla-1 La Salle - Universitat Ramon Llull 60.16%
top-tudelft-135 TUDELFT 56.27%
pol-uned2-5a8 LSI UNED 2 (angel, juan manuel & ana) 45.26%
top-imdea-1 IMDEA 45.24%
top-tudelft-246 TUDELFT 43.72%
pol-uned2-9a12 LSI UNED 2 (angel, juan manuel & ana) 42.24%
pol-uned2-1a4 LSI UNED 2 (angel, juan manuel & ana) 40.51%
top-sinai-5 SINAI - UJAEN 39.37%
top-sinai-4 SINAI - UJAEN 37.79%
top-sinai-2 SINAI - UJAEN 34.76%
top-sinai-3 SINAI - UJAEN 34.06%
top-sinai-1 SINAI - UJAEN 32.34%
pol-uned1-1y2 LSI UNED (tamara & jorge) 30.98%

Reports