{"id":34304,"date":"2025-02-10T10:46:22","date_gmt":"2025-02-10T09:46:22","guid":{"rendered":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/"},"modified":"2025-02-10T11:02:56","modified_gmt":"2025-02-10T10:02:56","slug":"oai-clarino-uib-no-lb-201803271","status":"publish","type":"language-resource","link":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/","title":{"rendered":""},"content":{"rendered":"


\n<\/p>\n

oai:clarino.uib.no:lb-201803271<\/identifier>2024-12-04T12:57:17Z<\/datestamp>FIN-CLARIN<\/setSpec><\/header>\n


\n


\nmetashareToCmdi.xsl remove_metashare_namespace.xsl<\/MdCreator>
\n2022-11-17<\/MdCreationDate>
\nurn:nbn:fi:lb-201803271<\/MdSelfLink>
\nclarin.eu:cr1:p_1361876010571<\/MdProfile><\/p>\n

<\/Header>Resource<\/ResourceType>http:\/\/urn.fi\/urn:nbn:fi:lb-201803272<\/ResourceRef><\/ResourceProxy>
\n<\/ResourceProxyList>
\n
\n<\/Resources>
\n
\n
\n
\n Helsinki Swahili -korpus 2.0 (HCS 2.0), ladattava annotoitu versio<\/resourceName>
\n Helsinki Corpus of Swahili 2.0 (HCS 2.0) Downloadable Annotated Version<\/resourceName>
\n The corpus is available for Download at Kielipankki – the Language Bank of Finland at http:\/\/urn.fi\/urn:nbn:fi:lb-201803272<\/p>\n

This is the downloadable version of Helsinki Corpus of Swahili 2.0 (HCS 2.0) Annotated Version, see http:\/\/urn.fi\/urn:nbn:fi:lb-2016011301 for more information.<\/p>\n

The Helsinki Corpus of Swahili 2.0 Annotated Version containing about 25 million words will be available in Kielipankki – the Language Bank of Finland in Download (https:\/\/korp.csc.fi\/download) for academic use. This means that students and staff of universities can use the corpus by simply logging in with their university credentials. Alumni have the option to apply for access via https:\/\/lbr.csc.fi.<\/p>\n

Access rights instructions: https:\/ \/www.kielipankki.fi\/support\/access (in Finnish: https:\/\/www.kielipankki.fi\/tuki\/kayttooikeudet\/).<\/p>\n

Instructions on how to access Kielipankki corpora: https:\/\/www.kielipankki.fi\/support\/corpus-location\/ (in Finnish: https:\/\/www.kielipankki.fi\/tuki\/aineiston-sijainti\/).<\/p>\n

The corpus contains various kinds of linguistic information attached to each token. The corpus was annotated using the Salama Tagger.<\/p>\n

Preparation of the material<\/p>\n

Most of the corpus material was retrieved from the Web. This method was used increasingly after texts in the Web became available. Only texts in news media and on open government pages were retrieved. Some types of texts, such as books, were scanned and proofread. Part of the oldest news material before the time of scanners in the 1980\u2019ies was manually typed.<\/p>\n

The corpus material has gone through a series of formatting and correction routines.<\/p>\n

1. Converting the text into ascii-format, required by the tagger. There is a wild variety of codes for describing diacritics in Web texts. These had to be formalized.
\n2. Proofreading and correcting the text with a speller.
\n3. Analyzing the proofread text for finding still remaining typos and possibly new words.
\n4. Constructing a correction program that automatically corrects such typos that can be safely corrected. More than 8000 such mistake types were identified.
\n5. New words found in corpus were added to the parser.
\n6. Texts were corrected using the constructed correction program.
\n7. Metadata in text files were formalized.
\n8. Texts were converted into sentence-per-line format.
\n9. Text within each file was randomly shuffled to mix the sentence order.<\/p>\n

The result of these routines comprises the Helsinki Corpus of Swahili 2.0 Not Annotated Version.<\/p>\n

The result of these routines was annotated with Salama Tagger, thus producing the Korp format of the corpus.<\/p>\n

Metadata were added to each file.<\/p>\n

Structure of the corpus<\/p>\n

HCS 2.0 contains the following types of material:<\/p>\n

Old material<\/p>\n

1. Books
\n2. News
\nNew material
\n1. Bunge
\n2. News<\/p>\n

Old material contains material before 2003. Much of this material is in Helsinki Corpus of Swahili 1.0. The big difference is, however, that while in the earlier corpus only sections of books were included, in the new corpus whole texts are included. The other difference is that while in the old corpus text sections are in the original order, in the new corpus sentences are randomly shuffled.<\/p>\n

Most of the new material consists of news texts from 2004-2015. The section \u2018Bunge\u2019 contains Hansards of the Tanzanian Parliament from the years 2004, 2005 and 2006. Metadata in the beginning of each file give more information. Also the names of the files give hints of the contents of the files.<\/p>\n

A word in the annotated corpus contains normally the following types of information:<\/p>\n

1. token
\n2. stem
\n3. part-of-speech
\n4. morphological description
\n5. gloss in English
\n6. syntactic tag
\n7. rest of verb description<\/p>\n

The last point concerns only verbs.<\/p>\n

Detailed license information: See Documentation section below.<\/description>
\n hcs-a-v2-dl<\/resourceShortName>
\n http:\/\/urn.fi\/urn:nbn:fi:lb-201803272<\/url>
\n NOT_DEFINED_FOR_V2<\/metaShareId>
\n http:\/\/urn.fi\/urn:nbn:fi:lb-201803271<\/identifier>
\n <\/identificationInfo>
\n
\n available-restrictedUse<\/availability>\n \n CLARIN_ACA-NC<\/licence>\n academic-nonCommercialUse<\/restrictionsOfUse>
\n attribution<\/restrictionsOfUse>
\n downloadable<\/distributionAccessMedium>
\n See Documentation section<\/attributionText>
\n academic<\/userNature>
\n
\n distributionRightsHolder<\/role>
\n
\n University of Helsinki<\/organizationName>
\n UHEL<\/organizationShortName>
\n
\n firstname.surname@helsinki.fi<\/email>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/distributionRightsHolderOrganization>\n <\/licenceInfo>\n
\n iprHolder<\/role>\n \n Arvi<\/surname>
\n Hurskainen<\/givenName>
\n male<\/sex>
\n
\n arvi.hurskainen@helsinki.fi<\/email>
\n <\/communicationInfo>
\n
\n affiliation<\/role>
\n
\n University of Helsinki<\/organizationName>
\n UHEL<\/organizationShortName>
\n
\n firstname.surname@helsinki.fi<\/email>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/affiliation>\n <\/personInfo>\n <\/iprHolderPerson>
\n <\/distributionInfo>
\n
\n contactPerson<\/role>\n \n The Language Bank of Finland<\/surname>
\n User support at CSC – IT Center for Science Ltd.<\/givenName>
\n unknown<\/sex>
\n
\n kielipankki@csc.fi<\/email>
\n <\/communicationInfo>
\n
\n affiliation<\/role>
\n
\n CSC – Tieteen tietotekniikan keskus Oy<\/organizationName>
\n CSC \u2014 IT Center for Science Ltd<\/organizationName>
\n CSC<\/organizationShortName>
\n Kielipankki<\/departmentName>
\n Language Bank of Finland<\/departmentName>
\n
\n kielipankki@csc.fi<\/email>
\n http:\/\/www.csc.fi\/english<\/url><\/p>\n

P.O. Box 405<\/address>\n

FI-02101<\/zipCode>
\n Espoo<\/city>
\n Finland<\/country>
\n +358 (0)9 457 2001<\/telephoneNumber>
\n +358 (0)9 457 2302<\/faxNumber>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/affiliation>\n <\/personInfo>\n <\/contactPerson>
\n
\n 2018-03-23<\/metadataCreationDate>
\n English<\/metadataLanguageName>
\n en<\/metadataLanguageId>
\n 2021-06-15<\/metadataLastDateUpdated>
\n attribution details fixed<\/revision>
\n
\n metadataCreator<\/role>\n \n Dieckmann<\/surname>
\n Ute<\/givenName>
\n unknown<\/sex>
\n
\n ute.dieckmann@helsinki.fi<\/email>
\n <\/communicationInfo>
\n
\n affiliation<\/role>
\n
\n FIN-CLARIN<\/organizationName>
\n FIN-CLARIN<\/organizationShortName>
\n University of Helsinki<\/departmentName>
\n
\n finclarin@helsinki.fi<\/email>
\n http:\/\/www.helsinki.fi\/fin-clarin<\/url><\/p>\n

PO Box 24 (Unioninkatu 40)<\/address>\n

00014<\/zipCode>
\n University of Helsinki<\/city>
\n Finland<\/country>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/affiliation>\n <\/personInfo>\n <\/metadataCreator>
\n
\n metadataCreator<\/role>\n \n Westerlund<\/surname>
\n Hanna<\/givenName>
\n
\n hanna.westerlund@helsinki.fi<\/email>
\n <\/communicationInfo>
\n
\n affiliation<\/role>
\n
\n FIN-CLARIN<\/organizationName>
\n FIN-CLARIN<\/organizationShortName>
\n University of Helsinki<\/departmentName>
\n
\n finclarin@helsinki.fi<\/email>
\n http:\/\/www.helsinki.fi\/fin-clarin<\/url><\/p>\n

PO Box 24 (Unioninkatu 40)<\/address>\n

00014<\/zipCode>
\n University of Helsinki<\/city>
\n Finland<\/country>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/affiliation>\n <\/personInfo>\n <\/metadataCreator>
\n <\/metadataInfo>
\n
\n
\n documentation<\/role>
\n Resource group page: http:\/\/urn.fi\/urn:nbn:fi:lb-2014032624<\/documentUnstructured>
\n <\/documentationUnstructured>
\n
\n documentation<\/role>
\n How to cite: https:\/\/www.kielipankki.fi\/viittaus\/?key=hcs-a-v2-dl&lang=en<\/documentUnstructured>
\n <\/documentationUnstructured>
\n
\n documentation<\/role>
\n
\n manual<\/documentType>
\n Helsinki Corpus of Swahili 2.0 (HCS 2.0) Manual<\/title><br \/>\n <url>https:\/\/www.kielipankki.fi\/corpora\/hcs2\/<\/url><br \/>\n <\/documentInfo><br \/>\n <\/documentationStructured><br \/>\n <documentationStructured ComponentId=\"clarin.eu:cr1:c_1361876010648\"><br \/>\n <role>documentation<\/role><br \/>\n <documentInfo ComponentId=\"clarin.eu:cr1:c_1353678848788\"><br \/>\n <documentType>other<\/documentType><br \/>\n <title xml:lang=\"en\">License information<\/title><br \/>\n <url>http:\/\/urn.fi\/urn:nbn:fi:lb-201806052<\/url><br \/>\n <\/documentInfo><br \/>\n <\/documentationStructured><br \/>\n <documentationStructured ComponentId=\"clarin.eu:cr1:c_1361876010648\"><br \/>\n <role>documentation<\/role><br \/>\n <documentInfo ComponentId=\"clarin.eu:cr1:c_1353678848788\"><br \/>\n <documentType>manual<\/documentType><br \/>\n <title xml:lang=\"en\">The Korp corpus input format<\/title><br \/>\n <author>Jyrki Niemi<\/author><br \/>\n <url>http:\/\/urn.fi\/urn:nbn:fi:lb-201806053<\/url><br \/>\n <keywords>VRT<\/keywords><br \/>\n <\/documentInfo><br \/>\n <\/documentationStructured><br \/>\n <\/resourceDocumentationInfo><br \/>\n <resourceCreationInfo ComponentId=\"clarin.eu:cr1:c_1355150532303\"><br \/>\n <resourceCreatorPerson ComponentId=\"clarin.eu:cr1:c_1355150532304\"><br \/>\n <role>resourceCreator<\/role>\n <personInfo ComponentId=\"clarin.eu:cr1:c_1349361150746\">\n <surname xml:lang=\"en\">Arvi<\/surname><br \/>\n <givenName xml:lang=\"en\">Hurskainen<\/givenName><br \/>\n <sex>male<\/sex><br \/>\n <communicationInfo ComponentId=\"clarin.eu:cr1:c_1352813745460\"><br \/>\n <email>arvi.hurskainen@helsinki.fi<\/email><br \/>\n <\/communicationInfo><br \/>\n <affiliation ComponentId=\"clarin.eu:cr1:c_1352813745462\"><br \/>\n <role>affiliation<\/role><br \/>\n <organizationInfo ComponentId=\"clarin.eu:cr1:c_1352813745461\"><br \/>\n <organizationName xml:lang=\"en\">University of Helsinki<\/organizationName><br \/>\n <organizationShortName xml:lang=\"en\">UHEL<\/organizationShortName><br \/>\n <communicationInfo ComponentId=\"clarin.eu:cr1:c_1352813745460\"><br \/>\n <email>firstname.surname@helsinki.fi<\/email><br \/>\n <\/communicationInfo><br \/>\n <\/organizationInfo><br \/>\n <\/affiliation>\n <\/personInfo>\n <\/resourceCreatorPerson><br \/>\n <\/resourceCreationInfo><br \/>\n <relationInfo ComponentId=\"clarin.eu:cr1:c_1355150532307\"><br \/>\n <relationType>IsVariantFormOf<\/relationType><br \/>\n <relatedResource ComponentId=\"clarin.eu:cr1:c_1355150532308\"><br \/>\n <targetResourceNameURI>Helsinki Corpus of Swahili 2.0 (HCS 2.0) Annotated Version http:\/\/urn.fi\/urn:nbn:fi:lb-2016011301<\/targetResourceNameURI><br \/>\n <\/relatedResource><br \/>\n <\/relationInfo><br \/>\n <corpusInfo ComponentId=\"clarin.eu:cr1:c_1355150532309\"><br \/>\n <resourceType>corpus<\/resourceType><br \/>\n <corpusMediaType ComponentId=\"clarin.eu:cr1:c_1355150532310\"><br \/>\n <corpusTextInfo ComponentId=\"clarin.eu:cr1:c_1355150532311\"><br \/>\n <mediaType>text<\/mediaType>\n <lingualityInfo ComponentId=\"clarin.eu:cr1:c_1355150532313\">\n <lingualityType>monolingual<\/lingualityType>\n <\/lingualityInfo>\n <languageInfo ComponentId=\"clarin.eu:cr1:c_1355150532314\"><br \/>\n <languageId>sw<\/languageId><br \/>\n <languageName>Swahili<\/languageName><br \/>\n <\/languageInfo><br \/>\n <sizeInfo ComponentId=\"clarin.eu:cr1:c_1353678848785\"><br \/>\n <size>25000000<\/size><br \/>\n <sizeUnit>tokens<\/sizeUnit><br \/>\n <\/sizeInfo><br \/>\n <\/corpusTextInfo><br \/>\n <\/corpusMediaType><br \/>\n <\/corpusInfo><br \/>\n<\/resourceInfo><\/Components><\/CMD><\/metadata><\/record><\/p>\n","protected":false},"template":"","categories":[],"tags":[],"language-resource-type":[],"language-resource-origin":[7558],"class_list":["post-34304","language-resource","type-language-resource","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.1 (Yoast SEO v27.1.1) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>- Spr\u00e5kbanken<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/\" \/>\n<meta property=\"og:locale\" content=\"nb_NO\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/\" \/>\n<meta property=\"og:site_name\" content=\"Spr\u00e5kbanken\" \/>\n<meta property=\"article:modified_time\" content=\"2025-02-10T10:02:56+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Ansl. lesetid\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutter\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/\",\"url\":\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/\",\"name\":\"- Spr\u00e5kbanken\",\"isPartOf\":{\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/#website\"},\"datePublished\":\"2025-02-10T09:46:22+00:00\",\"dateModified\":\"2025-02-10T10:02:56+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/#breadcrumb\"},\"inLanguage\":\"nb-NO\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.nb.no\/sprakbanken\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Resources from the resource bank\",\"item\":\"https:\/\/www.nb.no\/sprakbanken\/en\/resource-catalogue\/\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/#website\",\"url\":\"https:\/\/www.nb.no\/sprakbanken\/\",\"name\":\"Spr\u00e5kbanken\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.nb.no\/sprakbanken\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"nb-NO\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"- Spr\u00e5kbanken","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/","og_locale":"nb_NO","og_type":"article","og_url":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/","og_site_name":"Spr\u00e5kbanken","article_modified_time":"2025-02-10T10:02:56+00:00","twitter_card":"summary_large_image","twitter_misc":{"Ansl. lesetid":"4 minutter"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/","url":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/","name":"- Spr\u00e5kbanken","isPartOf":{"@id":"https:\/\/www.nb.no\/sprakbanken\/#website"},"datePublished":"2025-02-10T09:46:22+00:00","dateModified":"2025-02-10T10:02:56+00:00","breadcrumb":{"@id":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/#breadcrumb"},"inLanguage":"nb-NO","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-201803271\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.nb.no\/sprakbanken\/"},{"@type":"ListItem","position":2,"name":"Resources from the resource bank","item":"https:\/\/www.nb.no\/sprakbanken\/en\/resource-catalogue\/"}]},{"@type":"WebSite","@id":"https:\/\/www.nb.no\/sprakbanken\/#website","url":"https:\/\/www.nb.no\/sprakbanken\/","name":"Spr\u00e5kbanken","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.nb.no\/sprakbanken\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"nb-NO"}]}},"lang":"nb","translations":{"nb":34304,"en":34307},"pll_sync_post":[],"_links":{"self":[{"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/language-resource\/34304","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/language-resource"}],"about":[{"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/types\/language-resource"}],"wp:attachment":[{"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/media?parent=34304"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/categories?post=34304"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/tags?post=34304"},{"taxonomy":"language-resource-type","embeddable":true,"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/language-resource-type?post=34304"},{"taxonomy":"language-resource-origin","embeddable":true,"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/language-resource-origin?post=34304"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}