Retrieve classification and correspondence tables with the correspondenceTables package

Source: vignettes/Retrieve_classification_and_correspondence_tables.Rmd

This vignette provides information about the functions included in the correspondenceTables package used to extract classification and correspondence tables from CELLAR and FAO endpoints.

The main two functions used to exact tables are retrieveClassificationTable() and retrieveCorrespondenceTable(), for classification and correspondence tables respectively. Other functions (prefixList(), classEndpoint(), structureData() and correspondenceList()) are used to understand the structure of each classification and retrieve information needed as input of the two main functions.

Extract Classification Tables

The function retrieveClassificationTable() retrieves the classification table from the CELLAR and FAO repository, which can be used as an input to the newCorrespondenceTable and updateCorrespondenceTable functions. This function has five arguments:

  • prefix - the SPARQL instruction for a declaration of a namespace prefix.
  • endpoint - the SPARQL Endpoint: CELLAR or FAO.
  • conceptScheme - taxonomy of the SKOS object to be retrieved.
  • level - levels of the objects in the collection to be retrieved.
  • language - language of the table.
  • CSVout - preferred choice of output.

The values of the prefix, conceptScheme and level can be found using the classEndpoint() and structureData() functions (see below). The arguments level and language are optional and set as default as level=“ALL”, language=“en” and CSVout = “FALSE”.

The namespace prefix needed to execute the SPARQL query are automatically generated using the prefixList() function (see below).

Application of function retrieveClassificationTable()

To following code is used to retrieve the NACE2 classification from CELLAR:

endpoint = "CELLAR"
# obtain prefix and conceptscheme using classEndpoint()
classEndpoint(endpoint)[[1]][,1:4]
prefix = "nace2"
conceptScheme = "nace2"
language = "en"
CSVout = FALSE

# identify lower level using structureData()
level_dt = structureData(prefix, conceptScheme, endpoint, language)
level = level_dt[nrow(level_dt),2] 
level_dt; level

dt = retrieveClassificationTable(prefix, endpoint, conceptScheme, level, language, CSVout)

Other functions used to extract Classification Tables

Following, the other functions used to extract the required information for the retrieveClassificationTable() function.

Application of function prefixList()

This function automatically obtains a list the prefixes from CELLAR and FAO used in the retrieveClassificationTable() and retrieveCorrespondenceTable() functions to retrieve the required tables. The list will be updated automatically each time a new classification or correspondence classification is introduced in CELLAR or FAO repository.

endpoint = "CELLAR"
prefixList(endpoint)

Application of function classEndpoint()

The function returns a table with information needed to retrieve the classification table: 

 endpoint = "ALL"
 list_data = classEndpoint(endpoint)

Application of function structureData()

Each classification has a different level based on their structure. The function structureData() retrieves information about the level names, their hierarchy and the numbers of records for all the classification available in the repositories (CELLAR and FAO). The level informations needed as an optional arguments of the retrieveClassificationTable() function.

The function returns a table with the following columns:

The following code produce a list including the structure of each classification available in CELLAR and FAO.

## Obtain the structure of all classifications 
## CELLAR
data_CELLAR = list()
endpoint = "CELLAR"
#Get info to retrieve structure using classEndpoint()
list_data = classEndpoint("ALL")

#loop over all classification in CELLAR
for (i in 1:nrow(list_data$CELLAR)){
  prefix = list_data$CELLAR[i,1] 
  conceptScheme = list_data$CELLAR[i,2] 
  #language by default is English
  data_CELLAR[[i]] = structureData(prefix, conceptScheme, endpoint)
}
names(data_CELLAR) = list_data$CELLAR[,1] 

## FAO 
data_FAO = list()
endpoint = "FAO"
for (i in 1:nrow(list_data$FAO)){
  prefix = list_data$FAO[i,1]
  conceptScheme = list_data$FAO[i,2] 
  data_FAO[[i]] = structureData(prefix, conceptScheme, endpoint)
}
names(data_FAO) =  list_data$FAO[,1]

Extract Correspondence Tables

The function retrieveCorrespondenceTable() retrieves the correspondences table from the CELLAR and FAO repository, which can be used as an input to the newCorrespondenceTable and updateCorrespondenceTable functions. This function has four arguments:

  • prefix - the SPARQL instruction for a declaration of a namespace prefix
  • endpoint - the SPARQL Endpoint: CELLAR or FAO
  • ID_table - the ID of the correspondence table
  • language - language of the table
  • CSVout - preferred choice of output

The values of the prefix and ID_table can be found using the correspondenceList() function (see below). The argument language is optional and set as default as language=“en” and CSVout = “FALSE”.

The namespace prefix needed to execute the SPARQL query are automatically generated using the prefixList() function (see above).

Application of function retrieveCorrespondenceTable()

To following code is used to retrieve the correspondence table between NACE2 and PRODCOM2021 from CELLAR:

endpoint = "CELLAR"
prefix = "nace2"
ID_table = "NACE2_PRODCOM2021"
language = "en"
CSVout = FALSE
dt = retrieveCorrespondenceTable(prefix, endpoint, ID_table, language, CSVout)

To following code is used to retrieve a specific correspondence table between CPCv21 and ISIC4 from FAO:

endpoint = "FAO"
prefix = "CPCv21"
ID_table = "CPC21-ISIC4"
language = "en"
CSVout = FALSE
dt = retrieveCorrespondenceTable(prefix, endpoint, ID_table, language, CSVout)

Other functions used to extract Correspondence Tables

Following, the function used to extract the required information for the retrieveCorrespondenceTable() function.

Application of function correspondenceList()

The correspondenceList() provides an overview of all the available correspondence classification from CELLAR and FAO repository.

corr_list = correspondenceList("ALL")

#provides a table with all the correspondence tables for NACE2 in CELLAR
corr_list$CELLAR$`nace2:`

#provides a table with all the correspondence tables for CPC v21 in FAO
corr_list$FAO$`CPCv21:`