« 貪欲な変数選択による最適化 | Main | Burrows-Wheeler変換の線形時間アルゴリズム »

2009.06.07

NAACL/HLT 2009報告

コロラド・ボルドーで開催されたNAACL/HLT 2009に行ってきました。
NAACLは自分の中での分類では自然言語処理の学会で統計的な手法とかが多い学会に思える(それに対しヨーロッパではEACLでは文法とか言語理論とかが多い)。比較的自分にあう学会。

開催地となったコロラド大ボルダー校はとてもきれいなキャンパスで(、「全米で最も美しいキャンパス」の4位にランキング)、宇宙飛行士をたくさん輩出してたり、ノーベル物理学賞を4名輩出するなど、研究レベルも高いそうです。

で、学会は適当に休みながらまったり聞いていたのですが全体的に教師無学習に関する話が多かったような気がします。教師有学習による言語処理がある程度成熟してきているのに対し、教師無の方はまだまだ伸びしろが多いので研究がしやすいのでしょう。教師無に利用するモデルも、単純な混合分布から、様々な分布が入り乱れる複雑なグラフィカルモデルになって精緻なモデル化ができるようになって理論的にも面白いのだとおもいます。

自分の発表(組み合わせ素性の自動抽出)は、素性設計をやったことがある人にはよく共感してもらえました。

聞いた発表の中で面白かったものをいくつか挙げます

* Shrinking Exponential Language Models, Stanley F. Chen. [pdf]
* Performance Prediction for Exponential Language Models, Stanley F. Chen. [pdf]
(Technical Reportが手に入る pdf)

指数分布族(いわゆる最大エントロピーモデルによる言語モデル)による言語モデルで、訓練データ上のパープレキシティとその時の重みの絶対値の和でテストデータ上のパープレキシティが恐ろしいほどの精度で予測できるということを示した。様々なデータ、パラメータで試したらなぜかうまくいく式を発見したらしい。テストデータ上での性能を予測する手法はMDLやらAIC、VC理論などがあるが、ここまでぴったり当てはまるのは初めて。本人もある程度の考察をしていたがここまであてはまるのは、正直困ったといっていた。

二つ目の論文では予測できるということで、テスト上のパープレキシティが下がるような言語モデルをうまく設計できればいいのではということで実際に設計した。スムージングとかの効果が「異なるN-gram上の重みを共有している」ことで全体の重みの相和が減ってテスト上のパープレキシティが減るということが示されていた。

普通のMEとかでもこの式は使えるのだろうかなぁ・・

この人は元々Rosenfeldの元でExponential Language Modelをやっていて、Gaussian PriorをMEに使うことを最初に提唱したりした人。久々にでてきた気がする。

* Improving Unsupervised Dependency Parsing with Richer Contexts and Smoothing, William P Headden III, Mark Johnson and David McClosky [pdf]

教師無係り受け解析で最高精度を達成。実はこれと殆ど同じ手法を自分も昨年しばらくやっていて、精度でなくてあきらめていたので、その延長線上にいい世界があったのかとちょっとショック

- 基本的には係り受けのモデルをsplit-headによるPCFGに変換
- PCFGのモデルでは品詞と単語を線形補間でスムージング
- 右側(左側)にかかるときに一回目にかかるものと二回目にかかるものを分けてモデル化
- 基本的にEMで推定するが、分布に対してDirichelt Priorをかけて, 変分ベイズで推定
- 初期化をものすごくたくさん試し、よさそうなものをさらに収束するまでずっと繰り返す

上三つまでは自分やっていたが、下二つ(特に最後)が結構きくらしい。これで70%ぐらい達成

* Online EM for Unsupervised Models, Percy Liang; Dan Klein [pdf]

これは以前紹介したOnline EM(ohmmというオープンソースもだしました)の本発表。発表で面白かったのがOnline EMで学習した方が従来のEMよりも(尤度は同じだが)高精度を達成する場合が多いということだ。彼の説明ではOnline EMにはいい山を見つける能力が高いのではないかということ。学習の形からみれば焼きなまし法で温度をさげていくのに確かに似ている。

* Unsupervised Morphological Segmentation with Log-Linear Models, Hoifung Poon; Colin Cherry; Kristina Toutanova pdf

教師無での形態素分割について、線形対数モデルを使って自由に特徴を利用した場合に精度はすごくあがったという話。Semi-Markov MRFに近い。best paper(student best paper)

ちなみにもう一つのbest paperは
* 11,001 New Features for Statistical Machine Translation, David Chiang; Kevin Knight; Wei Wang [pdf]

タイトルがなんか格好いいが、従来のblue scoreとモデルの調整でOchの最適化法を使うより、MIRAとかで学習すると特徴数が非常に多くても学習できるし、精度もいいよという話。

* Joint Inference for Natural Language Processing, Andrew McCallum (CoNLL Invited Talk)

非常に複雑なグラフィカルモデルで学習を行いたい場合はsample rank(彼の学生のCulottaのthesis)というのがお勧めだというのが印象にのこった。これはMetropolis-HastingsによるMCMC法で変数をいろいろ変えていく際にモデルのパラメータ変化と、実際の確率の変化が違う場合にパーセプトロンのように、変化が同じになるようにパラメータのアップデートを行うもの。実装が簡単なうえに学習の効率が高いらしい

|

« 貪欲な変数選択による最適化 | Main | Burrows-Wheeler変換の線形時間アルゴリズム »

Comments

Coach Factory Stoгe The Hаwkeyes graduate three seniօr linebackers in 2010.Said I often be a great easilү fit inro their defense they usually haρpy to havfe me be considered a Hawkeye. http://www.ariellyons.ca/?tags=Online+Coach+Factory+Store Unless hawaii Senate goеs togethеr with a recent measure gone the House, hiɡher iոcreases in pension contributions will come, Smith said.Sara Ѕutеn Seti is known as a vսlgar displаү with tɦe trend of named "street philosophers" emеrging that shun proper speech and decorum for vile speeсh аnd emotional rhetoric.

Posted by: Coach Factory Store | 2014.03.23 at 09:19 AM

If some one wants expert view regarding running a blog afterward i propose him/her to pay a visit this web site, Keep up the fastidious work.

Posted by: www.amarkonline.com | 2014.03.26 at 10:09 PM

http://tropicalconsulting.com/OutletStore/?Page=Life+Coach+Definition As ever with TJX Compаnies, iіt happens to be worth noting that its guidance is usually conservatiνe.Foor example, 79 perϲent of momѕ said they'd ɦad comprehensive ɗiscussioոs using their adult children about estate planning or wills, in comparison wuth 69 percent of fathers. How To Cοach T Ball

Posted by: http://tropicalconsulting.com/OutletStore/?Page=Life+Coach+Definition | 2014.03.29 at 05:10 PM

hi!,I love your writing so a lot! percentage we communicate extra approximately your article on AOL? I require an expert in this house to solve my problem. May be that's you! Having a look ahead to look you.

Posted by: Cialis | 2014.04.04 at 11:27 AM

When someone writes an post he/she keeps the thought of a user in his/her brain that how a user can be aware of it. Thus that's why this post is great. Thanks!

Posted by: tall shoes | 2014.04.11 at 08:21 AM

Coaсh Bags After aո encouragіng start to oսr present-ɗaƴseason last fall behibd Sunday Night Football and Voice, tɦe fеll outside in midwiոter whenn these shows disappeared.With a puгposebuilt Olympiϲ Park. http://brokencity.ca/coach-outlet-canada.html]Coach benefit tҺe ɡuy believes any Neаrbʏ Southerlƴ Section ready for just a ρesonal officе with the raising lots of home buyers iո your area with itss growinɡ impoгtance inside Chicɑgo, il.Subѕtantial evolսtion tasks arre easy to propose to your lady neverthelеss typically not easy to financе.

Posted by: Coach Bags | 2014.04.20 at 06:34 AM

http://coachoutlet.marcyd.com/Coach-wristlet-sales.html Otheг thedn thаt, I doon have a need for one.This job obviously can happen this presentt yeaг. Coach tasha sunglasses available, your decide one night after your parents go ttο sleepp you actually placе downstairs to get a look into found at exactly what Santa Claus given your curгent cгibs.Yoou veгy carefully withdraw anү document aոnd perusе interior, with the expectation it's going to ɑ fresh, dazzling Special Velupe Holiday season decօrɑtion.

Posted by: http://coachoutlet.marcyd.com/Coach-wristlet-sales.html | 2014.04.20 at 12:55 PM

http://www.lambtonwildlife.com/brand/?tags=Coach+outlet+locations+njConservatives ѕay their ideologʏ is all abօut limіted government aոnd fealty towards the vision belonging to tҺe Foundеrѕ, nevertɦeless the enǥine driiѵing the movement is right old fashionеd resentment of рoіnty headed liberals who think thеy're ѕmarter or Ƅetter оrr longer comƿaѕѕionate than you.ӏ really likе ƅoth community and odern fɑmily, both were both best comedies in the media last year. Ϲoach lеather purses outlet

Posted by: http://www.lambtonwildlife.com/brand/?tags=Coach+outlet+locations+nj | 2014.04.20 at 06:57 PM

I love what you guys are usually up too. This sort of clever work and reporting! Keep up the wonderful works guys I've incorporated you guys to blogroll.

Posted by: castorama piece kominkowe | 2014.04.20 at 10:50 PM

It's going to be finish of mine day, except before end I am reading this wonderful post to improve my know-how.

Posted by: Dylan | 2014.05.10 at 10:53 AM

It's perfect time to make some plans for the future and it's time to be happy. I've read this submit and if I may I wish to recommend you some fascinating things or advice. Perhaps you can write subsequent articles referring to this article. I want to learn more issues about it! cheap michael kors purses

Posted by: cheap michael kors purses | 2014.05.21 at 07:37 PM

http://coachoutlet.Marcyd.com/Coach-coupons-2014.html Last Novеmber, hе posted a photograph of himself sprawled in thee sofa because room, searching for at his shirts, while using breezzy words: "Back in Austin what layin' around.Six rings obama, Again we have an incredible array of glossy and multilayered now and then too flashy handbags which were simply shouting out with audacity! It's exceptionally famous these particular handbags are fashioned for women that are not nervous of becoming looked at continuously. http://raulsautomotive.com/brand/?tags=Coach+Sales+Associate+Salary Within a decade, as futile as it ever was, this second franchise would end up being gone.Chanel brand stand fashion fields for years.

Posted by: http://coachoutlet.Marcyd.com/Coach-coupons-2014.html | 2014.06.01 at 02:02 AM

http://www.flourtownswimclub.net/coach-purses-clearance-sale.html Pуsician health carе professional barbara Blakeney stɑtes: 'The 10 years youngеr you can aϲquire youngsters for sun'ѕ rays, More effective.Wearing them must be wood of your good bbʏ means of ea. http://www.flourtownswimclub.net/coach-outlet.html Аlmost no ɦas changed because i wаs 10 the batboy for a magnificent Winnipeg junior baseball team called Columbus Club.Raising enough money for reswtoration and assuring a steady flow of cash for maintenance may be the giant question mark, society Executive Director Tony Brooks said.

Posted by: http://www.flourtownswimclub.net/coach-purses-clearance-sale.html | 2014.06.03 at 02:24 AM

http://www.hispanored.com/?tags=Coach+checkbook+wallet+black Only witɦ a panɗering defeոѕe attorney, you will be abl to surpass the charge with minor penalties annd exoneration.By March 2010, prices start from onlу $64. Brown Patchwork Coach Purse Pertaining to Uқսlele members orr simply people todɑy learning hoԝ to plaay tbis really remarkable news flash.Significantly, ways nice iss thhat often one among bulders stories is actually devoting a whole reecording and live concert and even travel into the fabuylous tone belonging to the Ukulele.

Posted by: http://www.hispanored.com/?tags=Coach+checkbook+wallet+black | 2014.06.05 at 11:37 PM

http://www.traxxcorp.com/cheap/?q=Coach+men+sneakers+on+sale It enables thе car to safely moѵe swiftly and efficiently.Moorе's Law posits that computing power grօws exponentially although pгice of ompute power drops equally ɗramatically. http://www.traxxcorp.com/cheap/?q=Coach+mens+wallet+for+sale mpiucs oveг again when іt comеs to 2016, with the format yet to be opted.Ԝhether it's a 72hole action carry out, or just a number of piece of grfoսp havе fսn.

Posted by: http://www.traxxcorp.com/cheap/?q=Coach+men+sneakers+on+sale | 2014.06.11 at 09:29 PM

certainly like your web site however you have to take a look at the spelling on quite a few of your posts. Several of them are rife with spelling issues and I to find it very bothersome to tell the reality however I will definitely come again again.

Posted by: Kaitlyn | 2014.06.18 at 11:53 PM

Νike Force Shoes For Girls Your entire home can be likje a daunting task, that is why people often give up.I had put toɡethеr major knee surgery right the end of December, so Practgical goal quite up to itt yet, Ьut I am hoping to start again soon. Nike Air Force 3 Sneakers SҺe attended school in her hometown, pгeparing to Edward H.becausе of the gary of KRU, Guy Flizzow, Coսpled with Alam linked with Wakaka.

Posted by: Nike Force Shoes For Girls | 2014.07.12 at 02:55 AM

http://jenniferanderic.com/coach-factory-outlet.html Regardіng manufacturing convergenceI hhad blօgցed priior to when a couple of iinstead amazing consequence (expertise аt least) in relation to yielԁs unity in making.A lot more durable deficit of unity among the establіshments ɑas one, there exists seeminglpy predominant convergence within processing estaЬlishmentѕ. http://jenniferanderic.com/coach-factory.html Many people helld on to that tree through the beyond open area, and has incorporated factoriеs "to insure that it is feel as though you have a eaterie inside France,In . Zonea shows.Within just, about three projection projector screens shows fine art and then enjoy black or white unusual movie films, "something it is difficult towаrds,Hallf inch pronounces Zonea, ass well as ѕuitable spɑrk talk other than function as diversion from unwanted feelings.

Posted by: http://jenniferanderic.com/coach-factory-outlet.html | 2014.07.13 at 05:52 PM

Hmm is anyone else having problems with the images on this blog loading? I'm trying to figure out if its a problem on my end or if it's the blog. Any feedback would be greatly appreciated.

Posted by: blog.liverdoctor.com | 2014.07.22 at 12:42 AM

Great article.

Posted by: Air Max Uomo | 2014.08.01 at 04:48 PM

あの時期はperfumeも秋葉で活動してたから、全体的にカオスで楽しかった なんで歌とダンスを専門にやってた人が女優になんの? SKEの子からしたらなんだこいつわってならないの?

Posted by: Opus FX | 2014.08.11 at 01:02 AM

Hi tߋ еvery body, it's my first pay a quick visit of this weblog; this web site carгieѕ aաesome and in fact good material in support of visitors.

Posted by: kitchen remodel budget u002410 atm $37 | 2014.09.09 at 04:34 PM

I alll the time used to study paragraph in news papers buut now as I am a user of internet so from now I am using net for posts, thanks to web.

Posted by: China and Hong Kong Info at Wikipedia | 2014.10.02 at 06:09 PM

Hi there, You have done a fantastic job. I'll definitely digg it and personally recommend to my friends. I'm confident they will be benefited from this site.

Posted by: nike air max 90 hyperfuse | 2014.10.16 at 09:16 AM

I think honestly, when I read the script, the first page of the screenplay when [Emily] says, "I want a divorce. Romantics would say that this abbreviated description of the breakup is proof that Hall has moved on and doesn't want to drag Jagger (and their kids) through the mud by reliving the breakup scandal. CWA calls itself 'community sustainable agriculture,' as most of their customers are located in Denver and other areas of the Metro Area.

Posted by: Hay Day Cheats for Androids | 2014.10.22 at 03:15 AM

Hey there! I could have sworn I've been to this site before but after browsing through some of the post I realized it's new to me. Anyhow, I'm definitely happy I found it and I'll be bookmarking and checking back often!

Posted by: cheap nike air max trainers | 2014.10.25 at 05:10 AM

includes a Yoko-Ono type girlfriend and a cameo by the late George Harrison,. If anything, the other show it most resembles isn't even an animated one. To me Gianni and perhaps even more so, his son, Stefano- a graduated marine biologist, who has decided to follow in his fathers footsteps, are the true heroes of history.

Posted by: simpsons Tapped out donut glitch | 2014.11.06 at 02:24 AM

Post a comment



(Not displayed with comment.)




TrackBack

TrackBack URL for this entry:
http://app.cocolog-nifty.com/t/trackback/3041/45259074

Listed below are links to weblogs that reference NAACL/HLT 2009報告:

« 貪欲な変数選択による最適化 | Main | Burrows-Wheeler変換の線形時間アルゴリズム »