Paieškos sistemos „Google“ naudojimo galimybės tiriant kalbos reiškinius

266 Languages and Culture | 81 SKAISTĖ ALEKSANDRAVIČIŪTĖ lietuvių kalbos institutas SEARCH SYSTEM GOOGLE Increasingly, Lithuanian linguists indicate in their articles that some kind of Internet search engine has been used in the collection of certain Russian data – from examples of usage to numerical frequency expressions. Google, the largest search engine, is often mentioned, but unfortunately it is not sufficiently adapted to study languages. It is therefore rather strange that it is generally confined to referring to Google as one of the sources, without indicating the method of search and without addressing certain features of the search engine and language that may make the search more difficult or difficult, refine or distort the results. The search method is important both for collecting usage samples and for studying the prevalence of various kalbas phenomena, where numerical expressions are important. This article discusses the principles of creating a search string, taking into account the peculiarities of the search engine and the Lithuanian language, which determine the success of the search. The article is based on a study of the relative frequency of new healthy words (hereinafter Ns) carried out in 2007-2008, the main objective of which is to establish the boundaries between rare, moderate and frequent Ns. The work was based on the assumption that the frequency reference points established during the study will later be used to evaluate the prevalence of other Ns in the usage. This could also be useful in language standardization activities, so the purpose of the article is to describe the way of creating the search string. The material of the research – 2005–2006 press and documents and (20 singular and 5 plural 2007 m. internete rasti Ns1 – iš viso 201. sąrašas labai įvairus: jame 33 veiks mažodžiai, 168 vardažodžiai, tarp kurių pasitaikė 25 trūktinos paradig mos nouns), is also in English, and others 1 list was compiled under the State Language Standardization, Use, Education and Dissemination 2006–2015 program using the Language Culture Department of the Lithuanian Language Institute’s project “Study of the Use of New Foreign Words in the Lithuanian Language: 1998–2008” kar toteka. s. alEKsaNDraVIčIŪTĖ. Possibilities of using the search engine Google to study linguistic phenomena of 267 language origin Ns. It is noteworthy that some of the words studied as Ns are included in the International Dictionary of Words (tŽZ 2001) and can be considered to be Tarot national words. The Internet is an extremely heterogeneous and constantly changing space. During the study, the web pages were not divided either by theme or by style (although this is possible in the Google search engine), because it was wanted to create a fairly bold picture of the prevalence of Ns in general usage. About the specific features of the online discourse this article talks somewhat generalized, emphasizes the conditionality of the mass, considerable attention is paid to possible errors and in many places is guided by linguistic instinct. The first part of the article presents the basic principles of search string composition, both general and specific to the specific study described. the second – indicates the ways of narrowing the search field and the reasons why it is necessary. 1. BASIC SEARCH STRENGTH CREATION PRINCIPLES Usually, a Google search is performed by typing the basic form of the word of interest in the search box. This method of search is partly sufficient for collecting the pupils of usage, but it is likely that a large part of them may go unnoticed, which is even less convenient for studying the frequency of words, where numerical expressions are important. The problem arises because Google's search engine partially understands that a typed word may have other forms and includes them in the results. but, first of all, the search algorithm is not completely clear to the ordinary user, it is not clear how many and what forms of the word will be among the results. secondly, the search system does not recognize parts of speech. For example, when entering the verb conjugation, the results include nouns of the same root, for example, when entering the word authorize, the system will also show texts that use the word authorization. For the latter two reasons, I prefer to do a more complex search, which is more useful for the research of the Chinese language. Various tests have shown that it is most appropriate to record the whole or most of the paradigm of word change. The search string is limited by the fact that Google’s search box can hold a maximum of 32 keywords. 268 Languages and Culture | 81 It is advisable to separate the forms of the word in quotation marks, because only then the text is understood by Google as an exact quotation. All forms of the word kaiak are connected by the conjunction or. In this way, the search string looks like this: „akreditavimas“ or „akreditavimo“ or „akreditavimui“ or „akreditavimą“ or „akreditavimu“ or „akreditavime“ or „akreditavimai“ or „akreditavimų“ or „akreditavimams“ or „akreditavimus“ or „akreditavimais“ or „akreditavimuose“ The above mentioned study of relative frequency of Ns was carried out using the Google search engine filter „pages in Lithuanian language“, which has already been criticized. The reason for this is that about two thirds of the information is lost, i.e. about as many examples of usage, because not all owners of Internet sites provide the linguistic metadata of the sites. but for relative word frequency studies this is quite sufficient. Moreover, without using this filter it is extremely difficult to distinguish between the English texts presented by the search engine, which impede the collection of examples of usage. Therefore, we think that searching in Lithuanian-language websites is more appropriate, it is only important that this restriction is consistently observed everywhere. Punctuation The Google search engine responds equally to all punctuation, even if it is in quotation marks and should be interpreted as an exact quotation. Thus, the search field "account" will be evaluated as two separate words and mixed with "account:as", "account as" or other similar cases. This feature of the search engine is important in two cases. Firstly, when Ns is made up of two words, the system does not distinguish between spelling variants such as ‘ex post’ and ‘exhpost’, which could be relevant in assessing the use of a hyphen. In these cases, the frequency results are combined into one, but when comparing the overall frequency of words rather than their individual spelling variants in the search, it is not necessary to distinguish between these forms. secondly, when studying the spelling variants of Ns with an apostrophe, we are faced with somewhat more complicated problems, because some Lithuanian endings coincide with: a) some English words (e.g. as – “as”, is – “is”, us – “us”); b) some Lithuanian words (conjunction o, prefix į, emoticons ei, ai); c) Lithuanian words written without diacritics („šveplai“) (is – „iš“, as – „aš“). s. alEKsaNDraVIčIŪTĖ. The possibilities of using the search engine Google to investigate language phenomena 269 It is very difficult to control the recent inaccuracies in the search, and the orthographic coincidence of Ns endings with Lithuanian words is even impossible. However, the cases where Ns is followed immediately by one of the abovementioned Lithuanian žodžiai, coinciding with the written expression of the change formant, are not so frequent as to have a significant effect on the results of the research. Selection of the search string forming changing forms When studying the frequency of noun Ns or their spelling variants, the whole paradigm of verb and number conjugation was entered into the search string in many cases. The exceptions were words with incomplete paradigms (20 singular and 5 plural nouns) and cases of polysemanticism (explained in the second part of the article). also due to orthographic coincidence with other forms of change and low prevalence, it is not worth including the vocalist separately. thus, it is possible to cover almost the entire paradigm of noun permutation. With regard to the conjugation of noun Ns, the problematic question of the genus should be discussed. Somewhere the feminine gender could be considered as a variant of Ns writing, for example, googlė / gūglė / google‘ė / googl‘ė; puzzle / puzzle / puzzle; Skype/Skype. These Ns have in common that they denote inanimate objects and therefore are not replaced by relatives in Lithuanian, and that their authentic forms have an unspoken but written ending -e at the end, e.g. google, puzzle, skype. In the study, it was observed that feminine gender forms are in other cases much rarer than masculine gender forms, which can be explained by the assumption of the optimality theory that “in the presence of a contradiction, the most marked category is chosen” (Vaicekauskienė 2007: 177). The masculine and feminine conjugation of adjectives and nouns denoting living objects were also not considered separate lemas, although there is a difference in meaning. loreta Vaicekauskienė, having examined Ns in Lithuanian in detail, notes that “[t]he cases where a noun can be used to name persons of both sexes, it seems that the masculine genus is initially generalized, from which, if necessary, the feminine genus form is sometimes formed” (Vaicekauskienė 2007: 175). Masculine and feminine forms were also not considered separate lemas in the preparation of Frequency of 270 Language Culture | 81 Dictionary of the Written Lithuanian Language (DDrlKŽ 1998: XIV–XV, XVI). This must be taken into account when creating search strings, as it is not possible to distinguish between coinciding masculine and feminine forms of the noun when working with the Google search engine. Verbs The case of verbs is more complicated, because the number of grammatical categories of the verb is much greater. As the length of Google’s search string is limited, as mentioned above, the question arises as to which forms of the verb are most representative in terms of frequency and should be included in the search string. Present Tense. It goes without saying that in order to cover a larger distribution of verbs, the most commonly used forms must be entered in the search string. for this we need to have data on the frequency of grammatical categories in Lithuanian language, preferably in electronic discourse (ED). Until now, ED as a separate language variant besides written and spoken language has been studied more extensively by asta ryklienė, but her chosen research methodology does not provide the necessary data on the distribution of grammatical categories in ED. The aim of this author’s work was to compare ED with written and spoken language (ryklienė 2001: 3), and to examine the quantitative differences of parts of speech and word forms in written language and ED by comparing fifty compendiums of the most commonly used words and word forms in these language variants (ryklienė 2001: 9). This article is based on the research of Vida Žilinskienė and Erika Rimkutė on the use of žodžiai forms in Lithuanian language. However, the works of these authors differ in that V. Žilinskienė, unlike E. rimkutė, when studying the frequency of word forms, compares two functional styles – the journalistic and the subjective. In view of the fact that ED language is closer to a journalistic than a subjective style, the author of this article relies on statistics of the frequency of journalistic style grammatical forms. The data of V. Žilinskienė and E. rimkutė are very similar – the direct verb encompasses 92.12% (Žilinskienė 2002: 110), 91.5% (Žilinskienė 2002: 111). (rimkutė 2006: 35) all distributions of the verb; personified verb forms – 55.48 per cent (Žilinskienė 2002: 110), 50.7 per cent (rimkutė 2006: 35); 15.85 per cent (Žilinskienė 2002: 111). It is true that E. rimkutė (2006: 35) indicates the prevalence of communicative forms in a slightly different way – according to her, communicative forms comprise 33.6 per cent of non-personalized verb forms, whose overall prevalence is 49.3 per cent. s. alEKsaNDraVIčIŪTĖ. Possibilities of using the search engine Google to study language phenomena 271 common noun and masculine plural noun of passive atsižvelgiant į pastaruosius duomenis, į paieškos eilutę parankiausia įtraukti visas tiesioginės nuosakos asmenuojamąsias veiksmažodžio formas, type participant (compare orthographic coincidence with common noun). All 26 grammatical forms (a total of 21 meaningful search words, because the third person of all times coincides) make up slightly more than half of the total verb distribution. Example search string: ‘baninti’ or ‘baninu’ or ‘banini’ or ‘banina’ or ‘baniname’ or ‘baninaaan te’ or ‘baninau’ or ‘baninai’ or ‘banino’ or ‘baninome’ or ‘baninote’ or ‘baninsiu’ or ‘baninsi’ or ‘banins’ or ‘baninsime’ or ‘baninsite’ or ‘banindavau’ or ‘banindavai’ or ‘banindavo’ or ‘banindavome’ or ‘baah nindavote’ As already mentioned earlier, the search engine Google does not pay attention to punctuation and spelling variants with an apostrophe in two separate words. Since the length of the Google search string is limited, it is possible to exclude past tense variations from the search string when studying the spelling variants of verbs with an apostrophe, since the past tense is the rarest of all minor verb tenses according to both V. Žilinskienė and E. rimkutė. Of course, if it is necessary to compare the frequency of different spelling variants of the same word, it would be possible to perform a broader search, i.e. to create several search strings. However, when comparing the frequency of words rather than their variations, the method discussed seems to be sufficient. it is only important to be consistent everywhere and apply the same search criteria to all verbs. Iš to, kas pasakyta šiame skyrelyje, aišku, kad paieškoje įmanoma aprėpti daugiau ar mažiau visą vardažodžių kaitybos paradigmą, o veiksmažodžių – only slightly more than half of the change paradigm, thus losing a solid basis for comparison. Of course, it would be possible to calculate some coefficient of frequency ratio of verbs and nouns, but this requires more detailed ED studies. For the time being, it would be inappropriate to compare the frequency of individual verbs and nouns. Thus, the different conjugation of nouns and verbs leads to the fact that in frequency studies I prefer to divide them into two groups. 272 Languages and Culture | 81 Spelling variants When studying usage, the Internet search method performs the opposite action than the textbook method, because when creating a search string, it is necessary to guess in advance what spelling variants each word may have. When searching for language phenomena on the Internet, authentic spelling variants of English kalāl bo origin should be avoided, and random occurrences in the search lines should be eliminated. During the aforementioned study it was observed that without doing this it is almost impossible to narrow the search field so that the results do not include English insertions or phrases that are considered quotations. For example, a search for the words baby or crazy returns a lot of texts that mention the titles of songs: Baby one more time, Crazy and so on. A similar case is funky, because this Ns is named not only a style of music, but also a formerly very popular Lithuanian music group. Nevertheless, when assessing the relative frequency of such words, these spelling variants must be taken into account, as they are quite commonly used. spelling variants are examined, which, based on linguistic instinct, at at appear to be possible. The orthographically adapted foreign word is accompanied by the operatively possible endings. When Ns is not orthographically adapted to mas, the endings are simply added to the unchanged form of Ns. Especially lazy are the cases when the unchanged form of Ns ends in a vowel, because when studying the spelling variants of Ns without an apostrophe, it is dropped (it turned out that smileas, skypeas, googleas and similar spelling variants are rare in usage). When studying language phenomena that have not yet been standardized, the question often arises where the boundary between a spelling error and a spelling variant lies. This limit is determined by the frequency characteristic. For example, in the aforementioned Ns study, spelling variants of Ns that do not reach the threshold of 10 use cases are not included in the calculation of the total frequency of the individual Ns and are considered as spelling errors – isolated errors that do not indicate general usage tendencies. true, this prina chip was applied only to words that exceeded the 1000 usage limit, reu closer – not. However, sometimes it may be significant that the language phenomenon under study was not found on the Internet at all or it did not reach the 10-case limit. The numerical expressions indicating the frequency of the spelling variants selected according to these principles are added together to obtain an approximate frequency of the word. the following table, where the processing of the search results may be made clearer. s. alEKsaNDraVIčIūtĖ. Paieškos sistemos Google naudojimo galimybės tiriant kalbos reiškinius 273 1 lENtElĖ. Duomenų kaupimo būdas NS ra šybos variantas Paieškos eilutė atskiram rašybos variantui NS rašybos varianto dažnumas puzlas „puzlas“ or „puzlo“ or „puzlui“ or „puzlą“ or „puzlu“ or „puzle“ or „puzlai“ or „puzlų“ or „puzlams“ or „puzlus“ or „puzlais“ or „puzluose“ puzlė „puzlė“ or „puzlės“ or „puzlei“ or „puzlę“ or „puzlėje“ 3210 or „puzlėms“ or „puzles“ or „puzlėmis“ or „puzlėse“ puzlis „puzlis“ or „puzlio“ or „puzliui“ or „puzlį“ or „puzliu“ or „puzlyje“ or „puzliai“ or „puzlių“ or „puzliams“ or 6430 „puzlius“ or „puzliais“ or „puzliuose“ puzzlė „puzzlė or „puzzlės“ or „puzzlėmis“ or „puzzlėse“ puzzlis „puzzlis“ or „puzzlio“ or „puzzlei“ or „puzzlę“ or „puzzlė 458 je“ or „puzzlėms“ or „puzzliui“ or „puzzlį“ or „puzzliu“ or „puzzlyje“ or „puzzliai“ or „puzzlių“ or „puzzliams“ or „puzzlius“ or „puzzliais“ or „puzzliuose“ pūzlė „pūzlė“ or „pūzlės“ or „pūzlei“ or „pūzlę“ or 711 „pūzlėje“ 239 or „pūzlėms“ or „pūzles“ or „pūzlėmis“ or „pūzlėse“ or „puzzlai“ or „puzzlų“ or „puzzlams“ or „puzzlus“ or „puzzlais“ puzzlas „puzzlas“ or „puzzlo“ or „puzzlui“ or „puzzlą“ or „puzz lu“ or „puzzluose“ pūzlis „pūzlis“ or „pūzlio“ or „pūzliui“ or „pūzlį“ or „pūzliu“ or „pūzlyje“ or „pūzliai“ or „pūzlių“ or „pūzliams“ or „pūzlius“ or „pūzliais“ or „pūzliuose“ puzzle‘as „puzzle’as“ or 315 „puzzle’o“ or „puzzle’ui“ or „puzzle’ą“ or „puzzle’u“ or „puzzle’e“ „puzzle’ais“ or or „puzzle’ai“ or „puzz le’ų“ or „puzzle’ams“ or „puzzle’us“ or 167 101 in puzzles 80 11 711 A similar table was drawn up for each Ns studied. The first column lists all studied spelling variants of Ns, ordered by frequency, because 274 Language Culture | Article 81 distances itself from the discussion of which spelling variant of Ns could be considered a title word (lemma). the second column shows the number of search strings for one spelling variant after narrowing the search field, i.e. after removing all multi-valued forms (see section 2). The third column indicates the frequency of each spelling variant (these data are only intermediate), which is then added to obtain an approximate frequency of the word under investigation. Only the latter data are relevant for the determination of the relative frequency limits of Ns (the data in the table were collected between March and April 2008). 2. Narrowing the search field The problem of English homographs and spelling variants with an apostrophe can be partly solved by using the logical function “except”, which in Google is represented by the sign “aan”. This means that the search will not be performed on web pages containing the word that follows this mark. Therefore, by putting one of the most common English words at the end of the search string and marking it with the above mark, the number of English occurrences can be greatly reduced. In the above mentioned study, one of the most commonly used words in English, are2 (is – present tense plural treang chiasis person), was used everywhere, but another choice is also possible. It is only necessary to draw attention to the fact that the English word for this purpose must be chosen with great care, because some of the most common English words may coincide orthographically with some Lithuanian words (English it, be), conjugation endings (English is, as, a, I) or cases of “sweet” spelling (English is, as, I). In addition, a large part of the most common English words (a, the, of) perform a service function and are often used in various names, which can also be found on Lithuanian web pages. The same restriction applies in all cases where the search string consists of an unchanged (authentic or adapted spelling) form of Ns, irrespective of the language from which a particular word may have reached the Lithuanian language. 2 the 500 Most commonly used Words in the English language: http://www.world-english. 2008. pp. 500. (accessed 2008-06-07) s. alEKsaNDraVIčIŪTĖ. Possibilities of using the search engine Google in the study of language phenomena 281 SOURCES Paieškos sistema www.google.lt the 500 Most commonly used Words in the English language: http://www. World-english.org/english500.htm tŽŽ 2001 – Tarptautinių žodžių žodynas, Vilnius: alma littera. VDU Textbook of Contemporary Lithuanian Language: http://donelaitis.vdu.lt lItEratūra 1998: L. Grumadienė, V. Žilinskienė. Dictionary of the Lithuanian Language, Vilnius: Lithuanian Language Institute. G a ive n i s K., Kei ny s s. 1990: Kalbotyros terminų žodynas, Kaunas: Šviesa. 1980: The Lexicology of the Lithuanian Language, Vilnius: Mokslas. Jakaitienė E. 2005: Leksikografija, Vilnius: Institute of Science and Encyclopedia Publishing. lKE 1999: Encyclopedia of the Lithuanian Language, Vilnius: Institute of Science and Encyclopedias Publishing. rimkutė E. 2002: Homoforms in contemporary Lithuanian texts. – Lituanistica 2(50), 86–101. rimkutė E. 2003: Typology of Morphological Polysemanticity. – Lituanistica 4(56), 60–78. rimkutė E. 2006: Restriction of morphological polysemanticity in computer text. Doctoral dissertation, Kaunas. rimkutė E., Grybinaitė a. 2004: The most common types of morphological polysemanticism in Lithuanian and their automatic unambiguity. – Language Studies 5, 74–78. ryklienė a. 2001: Electronic discourse: features of language and style. Abstract of doctoral dissertation, Vilnius. Vaicekauskienė l. 2007: New foreign words in Lithuanian language. Language Policy and Use, Vilnius: Lithuanian Language Institute. Žilinskienė V. 2002: Use of word forms in Lithuanian language according to subjective and publicistic styles. – Lituanistica 1(49), 106–116. Received 08/07/2008 282 Languages and Culture | 81 POSSIBILITIES OF USING tHE sEarcH sYstEM GOOGLE In lINGuIstIc rEsEarcH Summary the paper introduces the methodology which can be used to implement Google as a tool for data accumulation and storage and investigating the spread of a linguistic phenomenon in the internet. the method is described in the framework of an in vestigation aimed at identifying relative frequency count of new loanwords between 2007 and 2008. the first section of the article describes the main principles of fill ing in the slot of a query: general as well as specific for a particular descriptive re search. rather than entering only the main word form, a proposal has been launched to enter the whole grammatical paradigm of the word or its major part considering that in the Google query slot the maximum number of key words is 32. the second section identifies the ways to narrow down the search domain and reasons why it should be done. this type of research is bound to face a problem of word and wordform ambiguity. the term of ambiguity has been selected to avoid confusion of more specific terms like homonyms, homographs, homophones etc. Moreover, the explication of the above terms often emphasises the phonetic aspect of the word, which for internet search is irrelevant. the above terms are not suf ficiently precise for some cases of ambiguity, when, for example, the spelling of one word form coincides with another form of the same word. the method of search is instrumental in collecting various examples of lan guage use as well as in investigating their frequency. also the method of relative frequency of loanwords could be applied in language standardisation, e.g. in com piling lists of new loanwords which are considered grave mistakes. sKaIstĖ alEKsaNDraVIčIūtĖ lietuvių kalbos institutas P. Vileišio g. 5, lt10308 Vilnius [email protected]