BETWEEN FIXED COMMON-ANSWER POOLS, VERSION 1 The frequency-ranked Easy and Standard answer-pool data in difficulty-words.js is an adapted selection of wordfreq English data. The ranked source data and derived pool selection are licensed under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0): https://creativecommons.org/licenses/by-sa/4.0/ Legal code: https://creativecommons.org/licenses/by-sa/4.0/legalcode.en Copyright 2022 Robyn Speer. Modified for Between by filtering to the game's English headwords, ranking by source frequency, choosing fixed nested pools, protecting five explicitly listed ordinary Standard words, and excluding nine name-inflated answer candidates. See ../WORD_BANK.md and committed sources. The source authors do not endorse Between. This adaptation notice applies to the difficulty pool data, not to unrelated application code or to the separate full word-bank data, whose notices are in word-bank-license.txt. Original source: https://github.com/rspeer/wordfreq Data snapshot: wordfreq/data/small_en.msgpack.gz Git blob: b0a745f017d7504fca9dd4e5a6681f0682c561a7 SHA-256: f94a80cba6a3857b260d0666b5432bb7ea9b85315574dee9c306e87f61298247 SUBTLEX-US and SUBTLEX-UK are freely available data by Marc Brysbaert, Boris New, Walter van Heuven, Paweł Mandera, Emmanuel Keuleers et al.; see the preserved source citations and notice below. --- ORIGINAL WORDFREQ NOTICE --- wordfreq Copyright 2022 Robyn Speer # Attribution notes Robyn Speer must be credited as Robyn Speer, which is her maiden name, used on academic work. Crediting her as Elia Robyn Lake (her married name) will make the credit less effective, as it will not line up with other work. Crediting Robyn Speer by a different name than one of the above is a serious violation of the license, in which case you do not have permission to use, copy, or redistribute wordfreq. If you use wordfreq in academic work, you must cite it. See "Citing wordfreq" in README.md. # Included licenses `wordfreq` is freely redistributable under the Apache license (see `LICENSE.txt`), and it includes data files that may be redistributed under a Creative Commons Attribution-ShareAlike 4.0 license (). `wordfreq` contains data extracted from Google Books Ngrams () and Google Books Syntactic Ngrams (). The terms of use of this data are: Ngram Viewer graphs and data may be freely used for any purpose, although acknowledgement of Google Books Ngram Viewer as the source, and inclusion of a link to http://books.google.com/ngrams, would be appreciated. `wordfreq` also contains data derived from the following Creative Commons-licensed sources: - The Leeds Internet Corpus, from the University of Leeds Centre for Translation Studies () - Wikipedia, the free encyclopedia () - ParaCrawl, a multilingual Web crawl () It contains data from OPUS OpenSubtitles 2018 (), whose data originates from the OpenSubtitles project () and may be used with attribution to OpenSubtitles. It contains data from various SUBTLEX word lists: SUBTLEX-US, SUBTLEX-UK, SUBTLEX-CH, SUBTLEX-DE, and SUBTLEX-NL, created by Marc Brysbaert et al. (see citations below) and available at . I (Robyn Speer) have obtained permission by e-mail from Marc Brysbaert to distribute these wordlists in wordfreq, to be used for any purpose, not just for academic use, under these conditions: - Wordfreq and code derived from it must credit the SUBTLEX authors. - It must remain clear that SUBTLEX is freely available data. These terms are similar to the Creative Commons Attribution-ShareAlike license. Some additional data was collected by a custom application that watched the streaming Twitter API, in accordance with Twitter's Developer Agreement & Policy. This software gives statistics about words that were commonly used on Twitter; it does not display or republish any Twitter content, and does not contain any content from after Twitter's sale. --- ORIGINAL WORDFREQ CODE LICENSE NOTICE --- Copyright 2022 Robyn Speer Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. This license applies to the code only. See NOTICE.md for particular details about the licensing of the code and the data. --- ORIGINAL SOURCE CITATIONS --- ## Citing wordfreq If you use wordfreq in your research, please cite it! We publish the code through Zenodo so that it can be reliably cited using a DOI. The current citation is: > Robyn Speer. (2022). rspeer/wordfreq: v3.0 (v3.0.2). Zenodo. https://doi.org/10.5281/zenodo.7199437 The same citation in BibTex format: ``` @software{robyn_speer_2022_7199437, author = {Robyn Speer}, title = {rspeer/wordfreq: v3.0}, month = sep, year = 2022, publisher = {Zenodo}, version = {v3.0.2}, doi = {10.5281/zenodo.7199437}, url = {https://doi.org/10.5281/zenodo.7199437} } ``` ## Citations to work that wordfreq is built on - Bojar, O., Chatterjee, R., Federmann, C., Haddow, B., Huck, M., Hokamp, C., Koehn, P., Logacheva, V., Monz, C., Negri, M., Post, M., Scarton, C., Specia, L., & Turchi, M. (2015). Findings of the 2015 Workshop on Statistical Machine Translation. - Brysbaert, M. & New, B. (2009). Moving beyond Kucera and Francis: A Critical Evaluation of Current Word Frequency Norms and the Introduction of a New and Improved Word Frequency Measure for American English. Behavior Research Methods, 41 (4), 977-990. - Brysbaert, M., Buchmeier, M., Conrad, M., Jacobs, A.M., Bölte, J., & Böhl, A. (2011). The word frequency effect: A review of recent developments and implications for the choice of frequency estimates in German. Experimental Psychology, 58, 412-424. - Cai, Q., & Brysbaert, M. (2010). SUBTLEX-CH: Chinese word and character frequencies based on film subtitles. PLoS One, 5(6), e10729. - Davis, M. (2012). Unicode text segmentation. Unicode Standard Annex, 29. - Halácsy, P., Kornai, A., Németh, L., Rung, A., Szakadát, I., & Trón, V. (2004). Creating open language resources for Hungarian. In Proceedings of the 4th international conference on Language Resources and Evaluation (LREC2004). - Keuleers, E., Brysbaert, M. & New, B. (2010). SUBTLEX-NL: A new frequency measure for Dutch words based on film subtitles. Behavior Research Methods, 42(3), 643-650. - Kudo, T. (2005). Mecab: Yet another part-of-speech and morphological analyzer. - Lin, Y., Michel, J.-B., Aiden, E. L., Orwant, J., Brockman, W., and Petrov, S. (2012). Syntactic annotations for the Google Books Ngram Corpus. Proceedings of the ACL 2012 system demonstrations, 169-174. - Lison, P. and Tiedemann, J. (2016). OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016). - Ortiz Suárez, P. J., Sagot, B., and Romary, L. (2019). Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures. In Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-7) 2019. - ParaCrawl (2018). Provision of Web-Scale Parallel Corpora for Official European Languages. - van Heuven, W. J., Mandera, P., Keuleers, E., & Brysbaert, M. (2014). SUBTLEX-UK: A new and improved word frequency database for British English. The Quarterly Journal of Experimental Psychology, 67(6), 1176-1190.