[Wiki] Fwd: [Wiki-research-l] [Wikitech-l] Tech Talk: Hadoop and Beyond. An overview of Analytics infrastructure, Tuesday!

[h]

維基媒體幎床倧會將開始前期會有Hackathon想組織䞀個遠距跚國的掻動
目暙是讓參與者運甚珟有皋匏碌及資料䟆分析GLAM的圖片的點擊率及分䜈。
垌望倧家問䞀䞋所圚城垂是吊有人願意孞習䞊寊䜜的。


---------- Forwarded message ----------
From: h <hanteng-***@public.gmane.org>
Date: 2014-07-28 9:19 GMT+01:00
Subject: Re: [Wiki-research-l] [Wikitech-l] Tech Talk: Hadoop and Beyond.
An overview of Analytics infrastructure, Tuesday!
To: Research into Wikimedia content and communities <
wiki-research-l-RusutVdil2icGmH+5r0DM0B+***@public.gmane.org>
Cc: Wikimedia developers <wikitech-l-RusutVdil2icGmH+5r0DM0B+***@public.gmane.org>, "A mailing list
for the Analytics Team at WMF and everybody who has an interest in
Wikipedia and analytics." <analytics-RusutVdil2icGmH+5r0DM0B+***@public.gmane.org>


Dear wiki-research-l and wiki-tech-l members,

Specifically on Nuria Ruiz and Andrew Otto's talk on July 15th on the
NARA analytics pilot: Commons:GLAMwiki_Toolset_Project/NARA_analytics_pilot
<https://commons.wikimedia.org/wiki/Commons:GLAMwiki_Toolset_Project/NARA_analytics_pilot>,
I wonder whether it is possible to duplicate this for other GLAM
institutions so as to expand its global GLAM outreach.

I plan to compare/contrast two to four GLAM institutions that host
substantial Chinese collections in China, Taiwan, the U.K. I hope that it
can be turned into a hackathon event to recruit coders and researchers from
the Chinese-speaking regions.

Another benefit to duplicate NARA analytics pilot is to demonstrate
the possible data-research workflow using the data and infrastructure
provided by the Wikimedia Foundation.

I am not sure if the following list contain all the tasks involved and
the time needed to finish them (Please give me your estimate, THANKS):

# Identify images to be logged for visiting traffic.
# Get log data permission (Is it gonna be difficult?)
# Start logging
# Visualize using glam-metrics (http://glam-metrics.wmflabs.org/)
# Localize the stats report with Chinese locale/translation
# Customize glam-metrics so that more than one GLAM institution can be
compared.

Please also let me know if I miss anything. Many thanks.

Best,

Thanks for this. Forwarding to Analytics and Research for others who are
curious.
Pine

This Tech Talk will be starting in 30 minuets. Thanks!

Hello!
Please join Nuria Ruiz and Andrew Otto next Tuesday, July 15th at 10am

SF

time/5pm UTC
<

http://www.timeanddate.com/worldclock/fixedtime.html?msg=Analytics+Tech+Talk&iso=20140715T10&p1=224&am=30

for a 30 min tech talk. You can join our hangout or follow along on

https://plus.google.com/u/0/b/103470172168784626509/events/c53ho5esd0luccd09a1c30rlrmg

(please note that a link to join the hangout will be posted in the

comments

of this event just as it starts).
You can follow ask questions on IRC during the talk in #wikimedia-dev.
If you are not able to follow along live, a video recording will be

posted

here
<

https://plus.google.com/u/0/b/103470172168784626509/103470172168784626509/videos

,
to the MediaWiki YouTube channel immediately following the tech talk for
you to view at any time.
*Hadoop and Beyond. An overview of Analytics infrastructure*In this tech
talk we will be presenting the analytics infrastructure that we have
recently rolled out in production. By now probably everybody knows that
wikimedia hosts an instance of hadoop from which we are going to extract
pageview data in the near future. But .. how exactly does the data get
there?
We will go over the path that webrequest log data takes from varnish to
kafka (a distributed log buffer) to hadoop and the challenges of

deploying

this java-based infrastructure in production. We will also talk about

how

can we query the data with hive, an SQL-like interface. How can you set

up

this stack on vagrant to play with and, last but not least, how we used

https://commons.wikimedia.org/wiki/Commons:GLAMwiki_Toolset_Project/NARA_analytics_pilot

Thanks!

_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wiki-research-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

--
--
維基癟科 - 人人可線茯的自由癟科
通 過 郵 件 和 新 聞 組 癌 蚀 ﹙癌 蚀 請 甹 箔 文 本﹚ :
zh_wikipedia ﹙﹚ googlegroups ﹙﹚ com
news://news.gmane.org/gmane.org.user-groups.wikipedia.chinese
維 基 癟 科 矀 組 瀏 芜 èš‚ 閱 與 取 消 :
﹙ 新 ﹚ httpS://groups.google.com/forum/#forum/zh_wikipedia
httpS://groups.google.com/group/zh_wikipedia/subscribe
httpS://groups.google.com/group/zh_wikipedia/boxsubscribe
---
悚已蚂閱「Google 網䞊論壇」的「zh.wikipedia」矀組因歀我們特別傳送這封郵件通知悚。
劂芁取消蚂閱這個矀組䞊停止接收䟆自這個矀組的郵件請傳送電子郵件到 zh_wikipedia+unsubscribe-/***@public.gmane.org。
劂芁圚網路䞊查看這項蚎論請造蚪 https://groups.google.com/d/msgid/zh_wikipedia/CAOnynsaeaxWW54XT2Wx0L52p67UQa55mgCakpeGh%2BYWM7qEXQg%40mail.gmail.com。
劂需曎倚遞項請前埀https://groups.google.com/d/optout。