Class DatumLoader

Load data for a set of source IDs, date range, and aggregate level using either the listDatumUrl() or datumReadingUrl() URLs of SolarQueryApi (the /datum/list or /datum/reading endpoints).

This object is designed to be used once per query. After creating the object and optionally configuring any other settings, call DatumLoader#fetch to start loading the data. The returned Promise will be resolved once all data has been loaded.

Example

const filter = new DatumFilter();
filter.nodeId = 123;
// configure other filter settings here...

const results = await new DatumLoader(new SolarQueryApi(), filter).fetch();
// results is an array of Datum objects

Version

2.0.0

Hierarchy (view full)

Implements

Constructors

constructor

Properties

#callback #concurrency #includeTotalResultsCount #incrementalMode #pageSize #promise? #proxyUrl #queue? #readingsMode #results? #state #urlParameters api authBuilder? filter

Methods

#handleResults #loadData callback concurrency fetch includeTotalResultsCount incremental load paginationSize parameters proxyUrl readings requestor

Constructors

constructor

  • new DatumLoader(api, filter, authBuilder?): DatumLoader

  • Constructor.

    Parameters

    • api: default

      a URL helper for accessing node datum via SolarQuery

    • filter: DatumFilter

      the filter parameters to use

    • Optional authBuilder: AuthorizationV2Builder

      the auth builder to authenticate requests with; if not provided then only public data can be queried; when provided a pre-signed key must be available

    Returns DatumLoader

Properties

Private #callback

#callback: null | LoaderDataCallbackFn<Datum[]>

Private #concurrency

#concurrency: number

When > 0 then make one request that includes the total result count and first page of results, followed by parallel requests for the remaining pages.

Private #includeTotalResultsCount

#includeTotalResultsCount: boolean

Private #incrementalMode

#incrementalMode: boolean

When true then call the callback function for every page of data as it becomes available. Otherwise the callback function will be invoked only after all data has been loaded.

Private #pageSize

#pageSize: number

Private Optional #promise

#promise?: Promise<Datum[]>

Private #proxyUrl

#proxyUrl: null | string

An optional proxy URL to use instead of the host returned by the configured SolarQueryApi. This should be configured as an absolute URL to the proxy target, e.g. https://query.solarnetwork.net/1m.

Private Optional #queue

#queue?: Queue

A queue to use for parallel mode, when concurrency configured > 0.

Private #readingsMode

#readingsMode: boolean

When true then invoke the /datum/reading endpoint to load data, otherwise use /datum/list.

Private Optional #results

#results?: Datum[]

Private #state

#state: number

Private #urlParameters

#urlParameters: null | object

Protected Readonly api

api: default

The API instance to use.

Protected Optional Readonly authBuilder

authBuilder?: AuthorizationV2Builder

An authorization builder to use to make authenticated HTTP requests.

Readonly filter

filter: DatumFilter

The filter.

Methods

Private #handleResults

  • #handleResults(error, done, page?): void

  • Invoke the configured callback function.

    Parameters

    • error: undefined | Error

      an optional error

    • done: boolean

      true if there is no more data to load

    • Optional page: Pagination

      the incremental mode page

    Returns void

Private #loadData

  • #loadData(page, q?): void

  • Load a single page of data, starting at a specific offset.

    Parameters

    • page: Pagination

      the page to load

    • Optional q: Queue

      the queue to use

    Returns void

callback

  • callback(): null | LoaderDataCallbackFn<Datum[]>

  • Get the optional callback function.

    Returns null | LoaderDataCallbackFn<Datum[]>

    the current callback function or null if not defined

  • callback(value): this

  • Set the callback function, invoked after all data has been loaded or after every result page in incremental mode.

    The callback function will be passed a minimum of two arguments: an error and the results. In incremental mode, the callback will also be passed a boolean that will be true on that last page of data, and a Pagination that details which page the callback represents.

    Parameters

    Returns this

    this object

concurrency

  • concurrency(): number

  • Get the concurrency limit to use for parallel requests.

    Returns number

    the current concurrency value

  • concurrency(value): this

  • Set the concurrency limit to use for parallel requests.

    By default requests are not made in parallel (this property is configured as 0). Change to a positive number to enable parallel query mode.

    When parallel mode is enabled the loader will make one request that includes the total result count and first page of results, followed by parallel requests for any remaining pages based on that total result count and configured page size.

    Parameters

    • value: number

      the concurrency level to use, or Infinity for no limit

    Returns this

    this object

fetch

includeTotalResultsCount

  • includeTotalResultsCount(): boolean

  • Get the flag for requesting the total results count.

    By default the datum loader will not request the overal total result count when querying for data, as this speeds up queries. By setting this to true the total result count will be requested on the first query page.

    Returns boolean

    the total results count inclusion mode

  • includeTotalResultsCount(value): this

  • Set the flag for requesting the total results count.

    By default the datum loader will not request the overal total result count when querying for data, as this speeds up queries. By setting this to true the total result count will be requested on the first query page.

    Parameters

    • value: boolean

      the flag to include total results count

    Returns this

    this object

incremental

  • incremental(): boolean

  • Get the incremental mode for loading the data.

    Returns boolean

    true if incremental mode is enabled

  • incremental(value): this

  • Set incremental mode for loading the data.

    When incremental mode is enabled (set to true) then the callback function will be invoked for each result page that is loaded. The function will be passed a second boolean argument that will be set to true only on the last page of result data, and a third Pagination` object argument that details the starting offset of the page.

    When incremental mode is disabled (set to false, the default) then all result pages are combined into a single array and the callback will be invoked just once.

    @param value true to enable incremental mode

    Parameters

    • value: boolean

    Returns this

    this object

load

  • load(callback?): this

  • Initiate loading the data.

    As an alternative to configuring the callback function via the DatumLoader.callback method,a callback function can be passed as an argument to this function. That allows this function to be passed to things like queue.defer, for example.

    Parameters

    Returns this

    this object

paginationSize

  • paginationSize(): number

  • Get the result pagination size.

    Returns number

    the pagination size; defaults to 1000

  • paginationSize(value): this

  • Set the result pagination size.

    Parameters

    • value: number

      the pagination size to set

    Returns this

    this object

parameters

  • parameters(): null | object

  • Get the additional URL parameters.

    Returns null | object

    the URL parameters, or null

  • parameters(value): this

  • Set additional URL parameters.

    The parameters are set as object properties. If a property value is an array, multiple parameters for that property will be added.

    Parameters

    • value: null | object

      the URL parameters to include with the JSON request, or null to remove any existing parameters object

    Returns this

    this object

proxyUrl

  • proxyUrl(): null | string

  • Get the URL to a proxy to use for loading the data.

    Returns null | string

    the proxy URL

  • proxyUrl(value): this

  • Set the URL to a proxy to use for loading the data.

    This can be configured as an absolute URL to the proxy server to use instead of making requests directly to the URL returned by the configured SolarQueryApi. For example:

    Parameters

    • value: null | string

      the proxy URL to set, or null or an empty string to not use any proxy

    Returns this

    this object

readings

  • readings(): boolean

  • Get the readings mode for loading the data.

    Returns boolean

    true to return reading query, false for list query

  • readings(value): this

  • Set the readings mode for loading the data.

    When readings mode is enabled (set to true) then the /datum/reading endpoint will be invoked to load data.

    When readings mode is disabled (set to false, the default) then the /datum/list endpoint will be invoked to load data.

    Parameters

    • value: boolean

      true to return reading query, false for list query

    Returns this

    this object

Protected requestor

  • requestor<V>(url, signUrl?): ((cb) => void)

  • Create a URL fetch requestor.

    The returned function can be passed to d3.queue or invoked directly.

    Type Parameters

    • V

    Parameters

    • url: string

      the URL to request.

    • Optional signUrl: string

      the URL to sign (might be different to url if a proxy is used)

    Returns ((cb) => void)

    a function that accepts a callback argument

Settings

Member Visibility

Theme

On This Page

constructor#callback#concurrency#includeTotalResultsCount#incrementalMode#pageSize#promise#proxyUrl#queue#readingsMode#results#state#urlParametersapiauthBuilderfilter#handleResults#loadDatacallbackconcurrencyfetchincludeTotalResultsCountincrementalloadpaginationSizeparametersproxyUrlreadingsrequestor