Class DatumLoader

Load data for a set of source IDs, date range, and aggregate level using either the listDatumUrl() or datumReadingUrl() URLs of SolarQueryApi (the /datum/list or /datum/reading endpoints).

This object is designed to be used once per query. After creating the object and optionally configuring any other settings, call Tool.DatumLoader#fetch to start loading the data. The returned Promise will be resolved once all data has been loaded.

Example

const filter = new DatumFilter();
filter.nodeId = 123;
// configure other filter settings here...

const results = await new DatumLoader(new SolarQueryApi(), filter).fetch();
// results is an array of Datum objects

Hierarchy (View Summary)

JsonClientSupport<SolarQueryApi, Domain.Datum[]>
- DatumLoader

Implements

Loader<Domain.Datum[]>

Index

Constructors

constructor

new DatumLoader(
    api: SolarQueryApi,
    filter: DatumFilter,
    authBuilder?: AuthorizationV2Builder,
): DatumLoader
Constructor.
Parameters
- api: SolarQueryApi
  a URL helper for accessing node datum via SolarQuery
- filter: DatumFilter
  the filter parameters to use
- OptionalauthBuilder: AuthorizationV2Builder
  the auth builder to authenticate requests with; if not provided then only public data can be queried; when provided a pre-signed key must be available
Returns DatumLoader
Overrides JsonClientSupport.constructor
- Defined in src/main/tool/datumLoader.ts:116

Properties

`Protected` `Readonly`api

api: SolarQueryApi

The URL helper instance to use.

`Protected` `Optional` `Readonly`authBuilder

authBuilder?: AuthorizationV2Builder

An authorization builder to use to make authenticated HTTP requests.

`Readonly`filter

filter: DatumFilter

The filter.

Methods

callback

callback(): null | LoaderDataCallbackFn<Domain.Datum[]>
Get the optional callback function.

Returns null | LoaderDataCallbackFn<Domain.Datum[]>
the current callback function or null if not defined
- Defined in src/main/tool/datumLoader.ts:203
callback(value: null | LoaderDataCallbackFn<Domain.Datum[]>): this
Set the callback function, invoked after all data has been loaded or after every result page in incremental mode.

The callback function will be passed a minimum of two arguments: an error and the results. In incremental mode, the callback will also be passed a boolean that will be true on that last page of data, and a Pagination that details which page the callback represents.
Parameters
- value: null | LoaderDataCallbackFn<Domain.Datum[]>
  the callback function to use, or null to remove the existing callback function
Returns this
this object
- Defined in src/main/tool/datumLoader.ts:216

concurrency

concurrency(): number
Get the concurrency limit to use for parallel requests.

Returns number
the current concurrency value
- Defined in src/main/tool/datumLoader.ts:142
concurrency(value: number): this
Set the concurrency limit to use for parallel requests.

By default requests are not made in parallel (this property is configured as 0). Change to a positive number to enable parallel query mode.

When parallel mode is enabled the loader will make one request that includes the total result count and first page of results, followed by parallel requests for any remaining pages based on that total result count and configured page size.
Parameters
- value: number
  the concurrency level to use, or Infinity for no limit
Returns this
this object
- Defined in src/main/tool/datumLoader.ts:157

fetch

fetch(): Promise<Domain.Datum[]>
Initiate loading the data.

Returns Promise<Domain.Datum[]>
a Promise for the final results
Overrides JsonClientSupport.fetch
- Defined in src/main/tool/datumLoader.ts:418

includeTotalResultsCount

includeTotalResultsCount(): boolean
Get the flag for requesting the total results count.

By default the datum loader will not request the overal total result count when querying for data, as this speeds up queries. By setting this to true the total result count will be requested on the first query page.

Returns boolean
the total results count inclusion mode
- Defined in src/main/tool/datumLoader.ts:333
includeTotalResultsCount(value: boolean): this
Set the flag for requesting the total results count.

By default the datum loader will not request the overal total result count when querying for data, as this speeds up queries. By setting this to true the total result count will be requested on the first query page.
Parameters
- value: boolean
  the flag to include total results count
Returns this
this object
- Defined in src/main/tool/datumLoader.ts:345

incremental

incremental(): boolean
Get the incremental mode for loading the data.

Returns boolean
true if incremental mode is enabled
- Defined in src/main/tool/datumLoader.ts:272
incremental(value: boolean): this
Set incremental mode for loading the data.

When incremental mode is enabled (set to true) then the callback function will be invoked after each result page that is loaded, passing all results loaded thusfar. The function will be passed a second boolean argument that will be set to true only on the last page of result data, and a third Pagination` object argument that details the starting offset of the page.

When incremental mode is disabled (set to false, the default) then all result pages are combined into a single array and the callback will be invoked just once.
Parameters
- value: boolean
  true to enable incremental mode
Returns this
this object
- Defined in src/main/tool/datumLoader.ts:289

jitter

jitter(): number
Get the concurrency jitter value to use for parallel requests.

Returns number
the current concurrency jitter value (milliseconds); defaults to 150
- Defined in src/main/tool/datumLoader.ts:174
jitter(value: number): this
Set the concurrency jitter amount to use for parallel requests.

When parallel mode is enabled by setting concurrency() to a positive value, a random amount of "pause" time can be added before parallel requests are made by configuring this to a positive value. This can be helpful to avoid API rate limiting errors.
Parameters
- value: number
  the concurrency jitter amount to use, in milliseconds, or 0 to disable
Returns this
this object
- Defined in src/main/tool/datumLoader.ts:186

load

load(callback?: LoaderDataCallbackFn<Domain.Datum[]>): this
Initiate loading the data.

As an alternative to configuring the callback function via the DatumLoader.callback method,a callback function can be passed as an argument to this function. That allows this function to be passed to things like queue.defer, for example.
Parameters
- Optionalcallback: LoaderDataCallbackFn<Domain.Datum[]>
  a callback function to use; either this argument must be provided or the function must have already been configured via DatumLoader.callback
Returns this
this object
Implementation of Loader.load
- Defined in src/main/tool/datumLoader.ts:448

paginationSize

paginationSize(): number
Get the result pagination size.

Returns number
the pagination size; defaults to 1000
- Defined in src/main/tool/datumLoader.ts:304
paginationSize(value: number): this
Set the result pagination size.
Parameters
- value: number
  the pagination size to set
Returns this
this object
- Defined in src/main/tool/datumLoader.ts:312

parameters

parameters(): null | object
Get the additional URL parameters.

Returns null | object
the URL parameters, or null
- Defined in src/main/tool/datumLoader.ts:235
parameters(value: null | object): this
Set additional URL parameters.

The parameters are set as object properties. If a property value is an array, multiple parameters for that property will be added.
Parameters
- value: null | object
  the URL parameters to include with the JSON request, or null to remove any existing parameters object
Returns this
this object
- Defined in src/main/tool/datumLoader.ts:246

proxyUrl

proxyUrl(): null | string
Get the URL to a proxy to use for loading the data.

Returns null | string
the proxy URL
- Defined in src/main/tool/datumLoader.ts:389
proxyUrl(value: null | string): this
Set the URL to a proxy to use for loading the data.

This can be configured as an absolute URL to the proxy server to use instead of making requests directly to the URL returned by the configured SolarQueryApi. For example:
- https://query.solarnetwork.net
- https://query.solarnetwork.net/1m
Parameters
- value: null | string
  the proxy URL to set, or null or an empty string to not use any proxy
Returns this
this object
- Defined in src/main/tool/datumLoader.ts:403

readings

readings(): boolean
Get the readings mode for loading the data.

Returns boolean
true to return reading query, false for list query
- Defined in src/main/tool/datumLoader.ts:360
readings(value: boolean): this
Set the readings mode for loading the data.

When readings mode is enabled (set to true) then the /datum/reading endpoint will be invoked to load data.

When readings mode is disabled (set to false, the default) then the /datum/list endpoint will be invoked to load data.
Parameters
- value: boolean
  true to return reading query, false for list query
Returns this
this object
- Defined in src/main/tool/datumLoader.ts:374

`Protected`requestor

requestor<V>(
    url: string,
    signUrl?: string,
    delay?: number,
): (cb: LoaderDataCallbackFn<V>) => void
Create a URL fetch requestor.

The returned function can be passed to d3.queue or invoked directly.
Type Parameters
- V
Parameters
- url: string
  the URL to request.
- OptionalsignUrl: string
  the URL to sign (might be different to url if a proxy is used)
- Optionaldelay: number
  an optional number of milliseconds to sleep before initiating the request
Returns (cb: LoaderDataCallbackFn<V>) => void
a function that accepts a callback argument
Inherited from JsonClientSupport.requestor
- Defined in src/main/net/jsonClientSupport.ts:61

state

state(): DatumLoaderState
Get the loader state.

Returns DatumLoaderState
the state
- Defined in src/main/tool/datumLoader.ts:263

Class DatumLoader

Example

Hierarchy (View Summary)

Implements

Index

Constructors

Properties

Methods

Constructors

constructor

Parameters

Returns DatumLoader

Properties

Protected Readonlyapi

Protected Optional ReadonlyauthBuilder

Readonlyfilter

Methods

callback

Returns null | LoaderDataCallbackFn<Domain.Datum[]>

Parameters

Returns this

concurrency

Returns number

Parameters

Returns this

fetch

Returns Promise<Domain.Datum[]>

includeTotalResultsCount

Returns boolean

Parameters

Returns this

incremental

Returns boolean

Parameters

Returns this

jitter

Returns number

Parameters

Returns this

load

Parameters

Returns this

paginationSize

Returns number

Parameters

Returns this

parameters

Returns null | object

Parameters

Returns this

proxyUrl

Returns null | string

Parameters

Returns this

readings

Returns boolean

Parameters

Returns this

Protectedrequestor

Type Parameters

Parameters

Returns (cb: LoaderDataCallbackFn<V>) => void

state

Returns DatumLoaderState

Settings

On This Page

`Protected` `Readonly`api

`Protected` `Optional` `Readonly`authBuilder

`Readonly`filter

`Protected`requestor