Streaming language-specific Twitter data with optimal keywords
| العنوان: | Streaming language-specific Twitter data with optimal keywords |
|---|---|
| المؤلفون: | Tim Kreutz, Walter Daelemans |
| المصدر: | University of Antwerp Proceedings of the 12th Web as Corpus Workshop, May, 2020, Marseille, France |
| مصطلحات موضوعية: | Computer. Automation, Linguistics |
| الوصف: | The Twitter Streaming API has been used to create language-specific corpora with varying degrees of success. Selecting a filter of frequent yet distinct keywords for German resulted in a near-complete collection of German tweets. This method is promising as it keeps within Twitter endpoint limitations and could be applied to other languages besides German. But so far no research has compared methods for selecting optimal keywords for this task. This paper proposes a method for finding optimal key phrases based on a greedy solution to the maximum coverage problem. We generate candidate key phrases for the 50 most frequent languages on Twitter. Candidates are then iteratively selected based on a variety of scoring functions applied to their coverage of target tweets. Selecting candidates based on the scoring function that exponentiates the precision of a key phrase and weighs it by recall achieved the best results overall. Some target languages yield lower results than what could be expected from their prevalence on Twitter. Upon analyzing the errors, we find that these are languages that are very close to more prevalent languages. In these cases, key phrases that limit finding the competitive language are selected, and overall recall on the target language also decreases. We publish the resulting optimized lists for each language as a resource. The code to generate lists for other research objectives is also supplied. |
| ردمك: | 979-1-09-554668-9 |
| URL الوصول: | https://explore.openaire.eu/search/publication?articleId=dedup_wf_001::e880f9d72569b1ff5c2af69162436921 https://hdl.handle.net/10067/1773620151162165141 |
| حقوق: | OPEN |
| رقم الأكسشن: | edsair.dedup.wf.001..e880f9d72569b1ff5c2af69162436921 |
| قاعدة البيانات: | OpenAIRE |
| ردمك: | 9791095546689 |
|---|