• 講義要項やWebシラバスの記載内容は、登録された受講生の人数や理解度に応じて、授業開始後に変更となる可能性があります。

main start


開講年度 2016年度 開講箇所 基幹理工学部

担当教員 酒井 哲也
学期曜日時限 春学期  火3時限
科目区分 専門選択 配当年次 4年以上 単位数 2
使用教室 53-401教室 キャンパス 西早稲田(旧大久保)
科目キー 2603034029 科目クラスコード 01
授業で使用する言語 日本語
  コース・コード INFI41ZL
大分野名称 情報学
中分野名称 知的システム
小分野名称 一般
レベル 総仕上げ 授業形態 講義


最終更新日時:2016/01/25 15:13:23

副題 Foundations for Information Access Evaluation
授業概要 This course will be given in English.

This course will discuss evaluation methods for various information access tasks. Evaluation methods are very important because

"if you cannot measure it, you cannot improve it." The course will cover data and measures for information access evaluation,

methods for ensuring reliable evaluation experiments, and recent advances at international conferences for evaluating

information access.


授業の到達目標 Students will learn about how information access tasks are defined and evaluated to advance the state of the art in the

research community. Specifically, they will learn how to conduct evaluation experiments to verify the effectiveness of

information access systems. Students who have completed this course should be able to participate and write a nonrefereed
paper for evaluation conferences such as TREC and NTCIR.
事前・事後学習の内容 See below.

APR12 #01 Introduction
APR19 #02 Statistical significance, power, and effect sizes
APR26 #03 IR evaluation measures based on binary relevance
  [REVISION of #01-#03 and PREPARATION for #04]
MAY10 #04 <Students' presentations> Evaluation based on binary relevance
MAY17 #05 IR evaluation measures based on graded relevance
  [REVISION of #05 and PREPARATION for #06]
MAY24 #06 <Students' presentations> Evaluation based on graded relevance
MAY31 #07 Advanced IR and IA measures
  [REVISION of #07 and PREPARATION for #08]
JUN07 #08 <Students' presentations> T-tests and confidence intervals in practice [For 2016, this class is canceled.]
JUN14 #09 ANOVA and multiple comparison procedures
  [REVISION of #09 and PREPARATION for #10]
JUN21 #10 <Students' presentations> Multiple comparison procedures in practice
JUN28 #11 TREC history
  [REVISION of #11 and PREPARATION for #12]
JUL05 #12 <Students' presentations> Recent advances at TREC
JUL12 #13 NTCIR history
  [REVISION of #13 and PREPARATION for #14]
JUL19 #14 <Students' presentations> Recent advances at NTCIR [For 2016, this class is canceled.]

教科書 酒井哲也: 情報アクセス評価方法論 検索エンジンの進歩のために, コロナ社, 2015.
Sakai, T.: Information Access Evaluation: For the Progress of Search Engines (in Japanese)

(For non-Japanese students: )
成績評価方法 Students will be graded based on six Course Navi review sheets (after Lectures #03, #05, #07, #09, #11, #13) and one presentation (given in one of the following classes: #04, #06, #08, #10, #12, #14). Each review sheet is worth 0-10 points and a presentation is worth 0-40 points. Thus you can earn up to 6*10+40=100 points.

[Course Navi review sheets] (0-10 points * 6 times)
*Students must submit at least 4 valid review sheets in order to earn the credits for this course. Otherwise they will receive an F.*
A valid review sheet must:
(a) be submitted before the deadline (Monday after each Lecture);
(b) contain a one-time password announced during class; and
(c) contain an answer to the assignment question given at the end of class.

[Student presentation] once per student, 0-40 points
Slides (in pdf) must be submitted to by 1pm on the day before your presentation day. (Presentation schedule to be announced later). You must follow the formatting instructions (to be announced later) when preparing your slides. The filename of your pdf should be
e.g. FIAE20160101-1w111111.pdf
and the subject of your email should also contain this filename.

- Whether proper presentation slides are submitted by the above deadline: 10 points
- Content of the slides: 10 points
- Presentation time management: 10 points
- Quality of the presentation: 10 points
Students who fail to submit the slides or those who fail to show up on the presentation day will receive an F.


Students who submitted at least four valid review sheets AND gave a satisfactory presentation will be graded as follows, based on total points earned.
A+ (roughly top 0-20%)
A (roughly 20-50%)
B (roughly 50-80%)
C (roughly 80-100%).
備考・関連URL TREC online proceedings:
NTCIR online proceedings:


Copyright © Media Network Center,Waseda University 2006-2019.All rights reserved.