{"id":33297,"date":"2025-02-10T10:43:40","date_gmt":"2025-02-10T09:43:40","guid":{"rendered":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/"},"modified":"2025-03-26T11:51:07","modified_gmt":"2025-03-26T10:51:07","slug":"oai-clarino-uib-no-lb-2016011302","status":"publish","type":"language-resource","link":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/","title":{"rendered":""},"content":{"rendered":"


\n<\/p>\n

oai:clarino.uib.no:lb-2016011302<\/identifier>2025-03-03T09:30:30Z<\/datestamp>FIN-CLARIN<\/setSpec><\/header>\n


\n


\nmetashareToCmdi.xsl remove_metashare_namespace.xsl<\/MdCreator>
\n2022-11-17<\/MdCreationDate>
\nurn:nbn:fi:lb-2016011302 <\/MdSelfLink>
\nclarin.eu:cr1:p_1361876010571<\/MdProfile><\/p>\n

<\/Header>Resource<\/ResourceType>http:\/\/urn.fi\/urn:nbn:fi:lb-2016042801<\/ResourceRef><\/ResourceProxy>
\n<\/ResourceProxyList>
\n
\n<\/Resources>
\n
\n
\n
\n Helsinki Swahili -korpus 2.0 (HCS 2.0), ei annotoitu versio<\/resourceName>
\n Helsinki Corpus of Swahili 2.0 (HCS 2.0) Not Annotated Version<\/resourceName>
\n The Helsinki Corpus of Swahili 2.0 Not Annotated Version, consisting of plain text without linguistic codes, contains about 25 million words. The corpus is available in Kielipankki, the Language Bank of Finland, download location: http:\/\/urn.fi\/urn:nbn:fi:lb-2016042801 <\/p>\n

Preparation of the material
\nMost of the corpus material was retrieved from the Web. This method was used increasingly after texts in the Web became available. Only texts in news media and on open government pages were retrieved. Some types of texts, such as books, were scanned and proofread. Part of the oldest news material before the time of scanners in the 1980\u2019ies was manually typed.
\nThe corpus material has gone through a series of formatting and correction routines.
\n1.\tConverting the text into ascii-format, required by the tagger. There is a wild variety of codes for describing diacritics in Web texts. These had to be formalized.
\n2.\tProofreading and correcting the text with a speller.
\n3.\tAnalyzing the proofread text for finding still remaining typos and possibly new words.
\n4.\tConstructing a correction program that automatically corrects such typos that can be safely corrected. More than 8000 such mistake types were identified.
\n5.\tNew words found in corpus were added to the parser.
\n6.\tTexts were corrected using the constructed correction program.
\n7.\tMetadata in text files were formalized.
\n8.\tTexts were converted into sentence-per-line format.
\n9.\tText within each file was randomly shuffled to mix the sentence order.<\/p>\n

The result of these routines comprises the Helsinki Corpus of Swahili 2.0 Not Annotated Version.<\/p>\n

Metadata were added to each file.<\/p>\n

Structure of the corpus
\nHCS 2.0 contains the following types of material:
\nOld material
\n1.\tBooks
\n2.\tNews
\nNew material
\n1.\tBunge
\n2.\tNews<\/p>\n

Old material contains material before 2003. Much of this material is in Helsinki Corpus of Swahili 1.0. The big difference is, however, that while in the earlier corpus only sections of books were included, in the new corpus whole texts are included. The other difference is that while in the old corpus text sections are in the original order, in the new corpus sentences are randomly shuffled.<\/p>\n

Most of the new material consists of news texts from 2004-2015. The section \u2018Bunge\u2019 contains Hansards of the Tanzanian Parliament from the years 2004, 2005 and 2006. Metadata in the beginning of each file give more information. Also the names of the files give hints of the contents of the files.<\/description>
\n hcs-na-v2<\/resourceShortName>
\n http:\/\/urn.fi\/urn:nbn:fi:lb-2016042801<\/url>
\n NOT_DEFINED_FOR_V2<\/metaShareId>
\n http:\/\/urn.fi\/urn:nbn:fi:lb-2016011302<\/identifier>
\n <\/identificationInfo>
\n
\n available-unrestrictedUse<\/availability>\n \n CC-BY<\/licence>\n attribution<\/restrictionsOfUse>
\n downloadable<\/distributionAccessMedium>
\n http:\/\/urn.fi\/urn:nbn:fi:lb-2016042801<\/downloadLocation>
\n See Documentation section.<\/attributionText>\n \n licensor<\/role>\n \n Arvi<\/surname>
\n Hurskainen<\/givenName>
\n male<\/sex>
\n
\n arvi.hurskainen@helsinki.fi<\/email>
\n <\/communicationInfo>
\n
\n affiliation<\/role>
\n
\n University of Helsinki<\/organizationName>
\n UHEL<\/organizationShortName>
\n
\n firstname.surname@helsinki.fi<\/email>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/affiliation>\n <\/personInfo>\n <\/licensorPerson>\n
\n distributionRightsHolder<\/role>
\n
\n University of Helsinki<\/organizationName>
\n UHEL<\/organizationShortName>
\n
\n firstname.surname@helsinki.fi<\/email>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/distributionRightsHolderOrganization>\n <\/licenceInfo>\n
\n iprHolder<\/role>\n \n Arvi<\/surname>
\n Hurskainen<\/givenName>
\n male<\/sex>
\n
\n arvi.hurskainen@helsinki.fi<\/email>
\n <\/communicationInfo>
\n
\n affiliation<\/role>
\n
\n University of Helsinki<\/organizationName>
\n UHEL<\/organizationShortName>
\n
\n firstname.surname@helsinki.fi<\/email>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/affiliation>\n <\/personInfo>\n <\/iprHolderPerson>
\n <\/distributionInfo>
\n
\n contactPerson<\/role>\n \n The Language Bank of Finland<\/surname>
\n User support at CSC – IT Center for Science Ltd.<\/givenName>
\n unknown<\/sex>
\n
\n kielipankki@csc.fi<\/email>
\n <\/communicationInfo>
\n
\n affiliation<\/role>
\n
\n CSC – Tieteen tietotekniikan keskus Oy<\/organizationName>
\n CSC \u2014 IT Center for Science Ltd<\/organizationName>
\n CSC<\/organizationShortName>
\n Kielipankki<\/departmentName>
\n Language Bank of Finland<\/departmentName>
\n
\n kielipankki@csc.fi<\/email>
\n http:\/\/www.csc.fi\/english<\/url><\/p>\n

P.O. Box 405<\/address>\n

FI-02101<\/zipCode>
\n Espoo<\/city>
\n Finland<\/country>
\n +358 (0)9 457 2001<\/telephoneNumber>
\n +358 (0)9 457 2302<\/faxNumber>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/affiliation>\n <\/personInfo>\n <\/contactPerson>
\n
\n 2016-01-13<\/metadataCreationDate>
\n English<\/metadataLanguageName>
\n en<\/metadataLanguageId>
\n 2025-03-03<\/metadataLastDateUpdated>
\n relation to former roof page removed, relation to annotated version added<\/revision>
\n
\n metadataCreator<\/role>\n \n Bartis<\/surname>
\n Imre<\/givenName>
\n male<\/sex>\n Project Coordinator<\/position>\n
\n imre.bartis@helsinki.fi<\/email>
\n http:\/\/orcid.org\/0000-0002-9455-8771<\/url>
\n Helsinki<\/city>
\n Finland<\/country>
\n <\/communicationInfo>
\n
\n affiliation<\/role>
\n
\n FIN-CLARIN<\/organizationName>
\n FIN-CLARIN<\/organizationShortName>
\n University of Helsinki<\/departmentName>
\n
\n finclarin@helsinki.fi<\/email>
\n http:\/\/www.helsinki.fi\/fin-clarin<\/url><\/p>\n

PO Box 24 (Unioninkatu 40)<\/address>\n

00014<\/zipCode>
\n University of Helsinki<\/city>
\n Finland<\/country>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/affiliation>\n <\/personInfo>\n <\/metadataCreator>
\n
\n metadataCreator<\/role>\n \n Dieckmann<\/surname>
\n Ute<\/givenName>
\n unknown<\/sex>
\n
\n ute.dieckmann@helsinki.fi<\/email>
\n <\/communicationInfo>
\n
\n affiliation<\/role>
\n
\n FIN-CLARIN<\/organizationName>
\n FIN-CLARIN<\/organizationShortName>
\n University of Helsinki<\/departmentName>
\n
\n finclarin@helsinki.fi<\/email>
\n http:\/\/www.helsinki.fi\/fin-clarin<\/url><\/p>\n

PO Box 24 (Unioninkatu 40)<\/address>\n

00014<\/zipCode>
\n University of Helsinki<\/city>
\n Finland<\/country>
\n <\/communicationInfo>
\n <\/organizationInfo>
\n <\/affiliation>\n <\/personInfo>\n <\/metadataCreator>
\n <\/metadataInfo>
\n
\n
\n documentation<\/role>
\n How to cite: https:\/\/www.kielipankki.fi\/viittaus\/?key=urn:nbn:fi:lb-2016011302&lang=en<\/documentUnstructured>
\n <\/documentationUnstructured>
\n
\n documentation<\/role>
\n
\n other<\/documentType>
\n Resource group page (hcs 2.0)<\/title><br \/>\n <editor>FIN-CLARIN<\/editor><br \/>\n <url>http:\/\/urn.fi\/urn:nbn:fi:lb-2014032624<\/url><br \/>\n <documentLanguageName>English<\/documentLanguageName><br \/>\n <documentLanguageId>en<\/documentLanguageId><br \/>\n <\/documentInfo><br \/>\n <\/documentationStructured><br \/>\n <documentationStructured ComponentId=\"clarin.eu:cr1:c_1361876010648\"><br \/>\n <role>documentation<\/role><br \/>\n <documentInfo ComponentId=\"clarin.eu:cr1:c_1353678848788\"><br \/>\n <documentType>other<\/documentType><br \/>\n <title>License\/Lisenssi (hcs-na-v2, CC-BY)<\/title><br \/>\n <editor>FIN-CLARIN<\/editor><br \/>\n <url>http:\/\/urn.fi\/urn:nbn:fi:lb-2025022607<\/url><br \/>\n <documentLanguageName>English<\/documentLanguageName><br \/>\n <documentLanguageId>en<\/documentLanguageId><br \/>\n <\/documentInfo><br \/>\n <\/documentationStructured><br \/>\n <\/resourceDocumentationInfo><br \/>\n <relationInfo ComponentId=\"clarin.eu:cr1:c_1355150532307\"><br \/>\n <relationType>IsOriginalFormOf<\/relationType><br \/>\n <relatedResource ComponentId=\"clarin.eu:cr1:c_1355150532308\"><br \/>\n <targetResourceNameURI>Helsinki Corpus of Swahili 2.0 (HCS 2.0) Annotated Version http:\/\/urn.fi\/urn:nbn:fi:lb-2016011301<\/targetResourceNameURI><br \/>\n <\/relatedResource><br \/>\n <\/relationInfo><br \/>\n <corpusInfo ComponentId=\"clarin.eu:cr1:c_1355150532309\"><br \/>\n <resourceType>corpus<\/resourceType><br \/>\n <corpusMediaType ComponentId=\"clarin.eu:cr1:c_1355150532310\"><br \/>\n <corpusTextInfo ComponentId=\"clarin.eu:cr1:c_1355150532311\"><br \/>\n <mediaType>text<\/mediaType>\n <lingualityInfo ComponentId=\"clarin.eu:cr1:c_1355150532313\">\n <lingualityType>monolingual<\/lingualityType>\n <\/lingualityInfo>\n <languageInfo ComponentId=\"clarin.eu:cr1:c_1355150532314\"><br \/>\n <languageId>sw<\/languageId><br \/>\n <languageName>Swahili<\/languageName><br \/>\n <\/languageInfo><br \/>\n <modalityInfo ComponentId=\"clarin.eu:cr1:c_1355150532318\"><br \/>\n <modalityType>writtenLanguage<\/modalityType><br \/>\n <\/modalityInfo><br \/>\n <sizeInfo ComponentId=\"clarin.eu:cr1:c_1353678848785\"><br \/>\n <size>25000000<\/size><br \/>\n <sizeUnit>tokens<\/sizeUnit><br \/>\n <\/sizeInfo><br \/>\n <\/corpusTextInfo><br \/>\n <\/corpusMediaType><br \/>\n <\/corpusInfo><br \/>\n<\/resourceInfo><\/Components><\/CMD><\/metadata><\/record><\/p>\n","protected":false},"template":"","categories":[],"tags":[],"language-resource-type":[],"language-resource-origin":[7558],"class_list":["post-33297","language-resource","type-language-resource","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.1 (Yoast SEO v27.1.1) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>- Spr\u00e5kbanken<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/\" \/>\n<meta property=\"og:locale\" content=\"nb_NO\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/\" \/>\n<meta property=\"og:site_name\" content=\"Spr\u00e5kbanken\" \/>\n<meta property=\"article:modified_time\" content=\"2025-03-26T10:51:07+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Ansl. lesetid\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutter\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/\",\"url\":\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/\",\"name\":\"- Spr\u00e5kbanken\",\"isPartOf\":{\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/#website\"},\"datePublished\":\"2025-02-10T09:43:40+00:00\",\"dateModified\":\"2025-03-26T10:51:07+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/#breadcrumb\"},\"inLanguage\":\"nb-NO\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.nb.no\/sprakbanken\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Resources from the resource bank\",\"item\":\"https:\/\/www.nb.no\/sprakbanken\/en\/resource-catalogue\/\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.nb.no\/sprakbanken\/#website\",\"url\":\"https:\/\/www.nb.no\/sprakbanken\/\",\"name\":\"Spr\u00e5kbanken\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.nb.no\/sprakbanken\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"nb-NO\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"- Spr\u00e5kbanken","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/","og_locale":"nb_NO","og_type":"article","og_url":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/","og_site_name":"Spr\u00e5kbanken","article_modified_time":"2025-03-26T10:51:07+00:00","twitter_card":"summary_large_image","twitter_misc":{"Ansl. lesetid":"4 minutter"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/","url":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/","name":"- Spr\u00e5kbanken","isPartOf":{"@id":"https:\/\/www.nb.no\/sprakbanken\/#website"},"datePublished":"2025-02-10T09:43:40+00:00","dateModified":"2025-03-26T10:51:07+00:00","breadcrumb":{"@id":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/#breadcrumb"},"inLanguage":"nb-NO","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.nb.no\/sprakbanken\/ressurskatalog\/oai-clarino-uib-no-lb-2016011302\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.nb.no\/sprakbanken\/"},{"@type":"ListItem","position":2,"name":"Resources from the resource bank","item":"https:\/\/www.nb.no\/sprakbanken\/en\/resource-catalogue\/"}]},{"@type":"WebSite","@id":"https:\/\/www.nb.no\/sprakbanken\/#website","url":"https:\/\/www.nb.no\/sprakbanken\/","name":"Spr\u00e5kbanken","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.nb.no\/sprakbanken\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"nb-NO"}]}},"lang":"nb","translations":{"nb":33297,"en":33300},"pll_sync_post":[],"_links":{"self":[{"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/language-resource\/33297","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/language-resource"}],"about":[{"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/types\/language-resource"}],"wp:attachment":[{"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/media?parent=33297"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/categories?post=33297"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/tags?post=33297"},{"taxonomy":"language-resource-type","embeddable":true,"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/language-resource-type?post=33297"},{"taxonomy":"language-resource-origin","embeddable":true,"href":"https:\/\/www.nb.no\/sprakbanken\/wp-json\/wp\/v2\/language-resource-origin?post=33297"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}