Parsing Data

Parsing data (csv, tsv, psv)

dataPipe().fromCsv(contentOrUrlOrPath[, options]) methods provides a method to load data from commonly recognizable data formats like: CSV, TSV, PSV - fromCsv. Which has two parameters:

• contentOrUrlOrPath - a string parameter what can load data either from string content or an URL (a simple HTTP GET request more complicated can be specified with httpOptions) or path (works for NodeJS only)
• options - an optional configurable object with the following properties:
  • separator - default is ','
  • skipRows - is a number of rows where we have to skip before starting parsing CSV header
  • skipUntil - (line: string) => boolean - a callback function what is called for each line. When the function returns false. It stops skipping and starts analyzing this line as a header.
  • takeWhile - (line: string) => boolean - a callback function that is called each time before parsing a row. If function returns false - it stops parsing anywhere further.
  • parseFields - {fieldName: fieldType ('datetime' | 'number' | 'boolean')} - a list of fields what should be parsed as date, number or boolean
  • elementSelector - (item: [key: string]=>string) => [key: string]=>any - elementSelector is a call back function that is called each time a line is parsed as a map <string: string>. Use it for more advanced data transformations
  • httpOptions - it is used with http URL request. Following fields could be configured: method: 'GET' | 'POST' | 'PUT' | 'DELETE', httpHeaders: {}, body: {} ,