Reporting (FlowPDF Groovy Library 1.1.2.0 API)

com.cloudbees.flowpdf.components.reporting.Reporting

This module provides a component for Release Command Center (RCC) integration. Before this component created, implementation of RCC integration was very complex and hard to setup. This component provides an unified interface for RCC integrations using FlowPDF-Groovy toolkit.
Component state
Current state of this component is late alpha version. Drop 1. Currently, the state of this component allows you to implement CollectReportingData procedure for single payload-type per module. However, multiple payload are designed, but not implemented yet. This will be a goal for the Drop 2.
Terminology

DevOps Insight Center
DevOps Insight provides dashboards that give you insights into deployment and release activities over time. The ability to visualize this information as dashboards enables enterprises to understand the overall status of their processes, identify hotspots that need action, understand trends, and find opportunities for further improvement. DevOps Insight provides several dashboards as well as the ability to create custom dashboards. These dashboards include the Release Command Center dashboard, which provides a centralized view of all activities and processes related to a release. These include key metrics from the tools used across the end-to-end process, from work item tracking and build automation to test and operations tools. The Release Command Center dashboard helps you to understand how many user stories are planned, how many of them are "dev complete", how many are tested, what is the success rate of each of them, and other metrics that can help you better manage software production.
Release Command Center
Release Command Center is a part of Devops Insight Center and it shows a metrics that are gathered from different datasource of different types.
Data source
A system from where we're retrieving data for Devops Insight Center to shot them in Release Command Center. It could be, basically, any kind of third-party software. For example, Jenkins, Jira, ServiceNow, SonarQube, etc.
Report Object Type
Each kind of report has it's own object type. Report object type defines a place where collected data will be shown. For example, Jenkins datasource has report object type "build" for builds and "quality" for test reports. JIRA has 'feature' for improvement tickets, and "defect" for bug issues. ServiceNow has an "incident" report object type.
Payload
Payload is a data that represents single entity for each report object type. For example, if we're collecting reporting data from jenkins for HelloWorld project, we will have a payload object of a build type for each build that were already built. So, if we have 10 builds, we should send 10 payloads. Each payload of certain report object type has its own set of defined fields that should be sent. We're building payloads from the data, that is being retrieved from data source.
Records set
Records set is a raw data that is retrieved from data source. It should be an array of hashes.
Data set
Data set is an array of hashes (dicts) that is created from records set. In that case it should be a flat array of a flat hashes. So, each of hashes should have only a scalar values for its keys.
Payload set
Payload set is a set of payloads, that are already prepared to be sent to the reporting system.
Transform script
A script that is provided by user using procedure form or directly to be applied to each record in records set.
Metadata
An information that allows reporting system to know last reported object. It is being built from latest payload that was sent. Metadata is being stored as JSON in the property. The address of these property is could be default or set by user. Default place of property is depends on the context. For regular context it will be set to the current project. In context of schedule it will be stored in the schedule properties.
Data Flow

Datasource => Records Set => Transformed Records Set => Data Set => Payload Set
The key idea of reporting is:
- Read data from datasource (jira, jenkins, etc) about some metric (issues count, build status, etc).
- Create a set of payloads from retrieved dataset.
- Send it to EF instance.
- Write a fingerprint (we call it metadata) of latest payload and store it somewhere.
- On the next run, validate metadata to make sure that we need or need no of new reports.
What do you need to implement reporting?
To implement reporting you need to have the following things (if you have all of them, reporting implementation for your plugin is simple enough).
An unique key/metric/value of each record/payload.
This value will be your keypoint of reporting. This value should be unique and you should have a way of comparison between two values, to get which is later. For example, in Jenkins it is a build number. Build number is an unique value of each build inside a jenkins project. Let's say, that we did a successful reporting of set of jenkins builds, and latest was 42. Then, someone triggering a new build, and latest build number is 43 now. That is, we can easily compare 42 and 43 and understand, that if new build number is more than latest stored, we need to report.
An API that allows you to get latest record.
When it is possible to get a latest record by your criteria from datasource, you can avoid filtering full list of jobs.
How to implement reporting?
To get reporting one need to create a subclass of Reporting within your project and implement following methods:
- List> initialGetRecords(FlowPlugin plugin, int initialRecordsCount) This method will be called in case we don't have any metadata yet (on the first run). In a simple case it will just retrieve some number (specified as initialRecordsCount) of the objects from reporting system.
- Map getLastRecord(FlowPlugin plugin) In case we have the metadata, this method will be called on each schedule run to ensure that last record is fresh.
- List> getRecordsAfter(FlowPlugin plugin, Metadata metadata) If getLastRecord() returns new object (with a greater/fresher value than we have stored in a metadata), we should request all the new objects that have appeared in the reporting system.
Default assumptions

Metadata compare
Normally, your metadata will contain a single key (e.g. buildNumber, timestamp, etc.). Assuming that you have provided name(s) of the key in the 'payloadKeys', defaul implementation of the compareMetadata() will do all the checking for metadata freshness automatically. If your case is different and the default implementation is not suitable for you, override the compareMetadata() method.
Transforming report object into the payload.
DevOps Insight Server requires reports in a specific format (you should check Docs for your CloudBees CD server to know it for sure). Method buildPayloadSet by default will use given an object returned by data methods (initialGetRecords, getLastRecord, getRecordsAfter) as if it returns suitable object. You will probably want to override this method [buildPayloadSet()] to correctly map the values.
Authors:
CloudBees.

Properties Summary

Properties
Type	Name and description
`Metadata`	`currentMetadata` Instance for the metadata that is stored in CloudBees CD server
`static FWLog`	`fwLog` Internal logging instance
`int`	`initialRecordsCount` Default value for the count of objects to retrieve on first run (no metadata)
`boolean`	`isPreviewMode` If is set to true, no modificable operations will be performed (saving of metadata, sending report objects)
`MetadataFactory`	`metadataFactory` Metadata provider
`java.lang.String`	`metadataUniqueKey` Unique name of the metadata property, used to deffer metadata for different instances of CRD in the same project
`java.util.ArrayList<java.lang.String>`	`payloadKeys` List of the payload keys that are used to build the metadata
`java.lang.String`	`pluginName` Name of the plugin where component is defined
`FlowPlugin`	`pluginObject` Link to an instance of a plugin
`java.util.ArrayList<java.lang.String>`	`reportObjectTypes` Types of reports that are processed by this component
`Engine`	`reportingEngine` CloudBees CD Reporting APIs wrapper
`Transformer`	`transformer` Instance of the transformer that should process objects before they are passed to the buildPayloadSet

Methods Summary

Methods
Type Params	Return Type	Name and description
	`Dataset`	`buildDataset(FlowPlugin plugin, java.util.List<java.util.Map> records)` This method is used to convert records to the compatible representation that can be transformed to the payload
	`protected java.lang.String`	`buildMetadataLocation()` This method builds a path to the metadata property (context dependable)
	`Payloadset`	`buildPayloadSet(FlowPlugin plugin, Dataset records)` Placeholder for the buildPayloadSet method
	`void`	`collectReportingData()` Entry method for the component process.
	`int`	`compareMetadata(Metadata currentMetadata, Metadata newMetadata)` This method is responsible for comparing current stored values and the one that was built from the result of getLastRecord()
	`java.util.Map<java.lang.String, java.lang.Object>`	`getLastRecord(FlowPlugin plugin)` This method is used to check freshness of current metadata.
	`java.util.List<java.util.Map<java.lang.String, java.lang.Object>>`	`getRecordsAfter(FlowPlugin plugin, Metadata metadata)` If current metadata is older than the record received by the getLastRecord(), this method will be called to get new records.
	`Component`	`init(java.util.Map<java.lang.String, java.lang.Object> configuration, FlowPlugin plugin)` Initialization of the component.
	`protected Transformer`	`initTransformer(java.lang.String transformScriptValue)`
	`java.util.List<java.util.Map<java.lang.String, java.lang.Object>>`	`initialGetRecords(FlowPlugin plugin, int initialRecordsCount)` This method will be called if there is no metadata stored for current Reporting configuration and context.
	`boolean`	`isEFComponent()` If a parameter named 'transformScript' was found and contains a closure an instance of the Transformer will be initialized to process report objects.
	`boolean`	`isPreview()` Getter for the isPreviewMode
	`Dataset`	`newDataset(java.util.ArrayList<java.lang.String> reportObjectTypes, java.util.List<Data> data)` Shortcut to create a dataset from given data
	`MetadataFactory`	`newMetadataFactory()`
	`Payloadset`	`newPayloadset(java.util.ArrayList<java.lang.String> reportObjectTypes, java.util.ArrayList<Payload> payloads = [])` Shortcut to create a payloadset from given payloads
	`Payloadset`	`prepareAndValidatePayloads(Payloadset payloadset)` This method is responsible for validating the payloads prepared to be sent to the CloudBees CD server
	`Payload`	`prepareAndValidateSinglePayload(Payload payload, Engine reportingEngine)` This method compares the payload with format defined on CloudBees CD server
	`Dataset`	`rawRecordsToDataset(java.util.List<java.util.Map<java.lang.String, java.lang.Object>> records)` Converts objects returned from the reporting system to a Data
	`static java.lang.Object`	`validateAndConvertRow(java.lang.String fieldName, java.lang.Object rowValue, java.util.Map<java.lang.String, java.lang.Object> fieldDefinition)` This method validates that the value for given key corresponds with the type that is defined on the CloudBees server.

Inherited Methods Summary

Inherited Methods
Methods inherited from class	Name
`class Component`	`init, isEFComponent`
`class java.lang.Object`	`java.lang.Object#wait(long, int), java.lang.Object#wait(long), java.lang.Object#wait(), java.lang.Object#equals(java.lang.Object), java.lang.Object#toString(), java.lang.Object#hashCode(), java.lang.Object#getClass(), java.lang.Object#notify(), java.lang.Object#notifyAll()`

- Property Detail
  - Metadata currentMetadata
    
    Instance for the metadata that is stored in CloudBees CD server
  - static FWLog fwLog
    
    Internal logging instance
  - int initialRecordsCount
    
    Default value for the count of objects to retrieve on first run (no metadata)
  - boolean isPreviewMode
    
    If is set to true, no modificable operations will be performed (saving of metadata, sending report objects)
  - MetadataFactory metadataFactory
    
    Metadata provider
  - java.lang.String metadataUniqueKey
    
    Unique name of the metadata property, used to deffer metadata for different instances of CRD in the same project
  - java.util.ArrayList<java.lang.String> payloadKeys
    
    List of the payload keys that are used to build the metadata
  - java.lang.String pluginName
    
    Name of the plugin where component is defined
  - FlowPlugin pluginObject
    
    Link to an instance of a plugin
  - java.util.ArrayList<java.lang.String> reportObjectTypes
    
    Types of reports that are processed by this component
  - Engine reportingEngine
    
    CloudBees CD Reporting APIs wrapper
  - Transformer transformer
    
    Instance of the transformer that should process objects before they are passed to the buildPayloadSet
- Method Detail
  - Dataset buildDataset(FlowPlugin plugin, java.util.List<java.util.Map> records)
    
    This method is used to convert records to the compatible representation that can be transformed to the payload
    Parameters:
    plugin - CloudBees CD plugin instance. Useful if you need to make additional requests
    records - Raw records from the reporting system
    Returns:
    Dataset instance - compatible representation that can be transformed to the payload
  - protected java.lang.String buildMetadataLocation()
    
    This method builds a path to the metadata property (context dependable)
    Returns:
    path to the metadata property
  - Payloadset buildPayloadSet(FlowPlugin plugin, Dataset records)
    
    Placeholder for the buildPayloadSet method
    Parameters:
    plugin - instance of a plugin
    records - set of records retrieved from the reporting system
    Returns:
    set of records we can send to the DevOpsInsightServer
  - void collectReportingData()
    
    Entry method for the component process.
  - int compareMetadata(@org.jetbrains.annotations.NotNull Metadata currentMetadata, @org.jetbrains.annotations.NotNull Metadata newMetadata)
    
    This method is responsible for comparing current stored values and the one that was built from the result of getLastRecord() Note: Result of all getter functions, that return an array reference should be ordered from older to newer. This will be checked during validation and procedure will fail if records are not sorted in proper order. For this validation function that has been written for metadata comparison by user will be used. For example, if you have a set of builds with 10, 11, and 48, they should go in the following order: [10, 11, 48], otherwise procedure will raise an exception.
    Parameters:
    currentMetadata - Metadata that is stored on CloudBees CD server for current configuration and context
    newMetadata - Metadata that was build from the result of getLastRecord()
    Returns:
    1 if newMetadata contains values never the currentMetadata, 0 if they are equal
  - @org.jetbrains.annotations.Nullable java.util.Map<java.lang.String, java.lang.Object> getLastRecord(FlowPlugin plugin)
    
    This method is used to check freshness of current metadata. Values of this record will be compared to current metadata.
    Parameters:
    plugin - FlowPlugin instance to request the values
    Returns:
    Raw object from the reporting system that will be used to build new metadata for comparing.
  - @org.jetbrains.annotations.Nullable java.util.List<java.util.Map<java.lang.String, java.lang.Object>> getRecordsAfter(FlowPlugin plugin, Metadata metadata)
    
    If current metadata is older than the record received by the getLastRecord(), this method will be called to get new records.
    Parameters:
    plugin - FlowPlugin instance to request the values
    metadata - current metadata. Use metadata.getValue() to get current stored data.
    Returns:
    List of raw objects from the reporting system that will be converted to a List to be converted to a Dataset
  - Component init(@org.jetbrains.annotations.NotNull java.util.Map<java.lang.String, java.lang.Object> configuration, @org.jetbrains.annotations.Nullable FlowPlugin plugin)
    
    Initialization of the component. Configuration keys are:
    - reportObjectTypes Types of reports that are processed by this component
    - payloadKeys List of the payload keys that are used to build the metadata
    - metadataUniqueKey Unique name of the metadata property, used to deffer metadata for different instances of CRD in the same project
    - initialRecordsCount Default value for the count of objects to retrieve on first run (no metadata)
    - previewMode If is set to true, no saving of metadata and sending the report objects will not be performed.
    Parameters:
    configuration - Configuration for this reporting component
    plugin - - instance of the plugin will be passed to perform an advanced tasks inside of the Component.
    Returns:
    Reporting component
  - @org.jetbrains.annotations.Nullable /** If a parameter named 'transformScript' was found and contains a closure an instance of the {@link Transformer} * will be initialized to process report objects. */ protected Transformer initTransformer(java.lang.String transformScriptValue)
  - @org.jetbrains.annotations.Nullable java.util.List<java.util.Map<java.lang.String, java.lang.Object>> initialGetRecords(FlowPlugin plugin, int initialRecordsCount)
    
    This method will be called if there is no metadata stored for current Reporting configuration and context. Can return null if there is no objects in the reporting system.
    Parameters:
    plugin - FlowPlugin ancestor to get the values
    initialRecordsCount - - see the documentation for Reporting.getInitialRecordsCount
    Returns:
    List of raw objects from the reporting system that will be converted to a List to be converted to a Dataset
  - boolean isEFComponent()
    
    If a parameter named 'transformScript' was found and contains a closure an instance of the Transformer will be initialized to process report objects.
  - boolean isPreview()
    
    Getter for the isPreviewMode
    Returns:
    true if reporting was instantiated with a 'previewMode'
  - Dataset newDataset(@org.jetbrains.annotations.NotNull java.util.ArrayList<java.lang.String> reportObjectTypes, java.util.List<Data> data)
    
    Shortcut to create a dataset from given data
    Parameters:
    reportObjectTypes - - types that this Dataset contains
    data - List of Data to be packet into the Dataset
    Returns:
    initializedDataset with given Data
  - MetadataFactory newMetadataFactory()
  - Payloadset newPayloadset(java.util.ArrayList<java.lang.String> reportObjectTypes, java.util.ArrayList<Payload> payloads = [])
    
    Shortcut to create a payloadset from given payloads
    Parameters:
    reportObjectTypes - - types that this Payloadset contains
    data - List of Payload to be packet into the Payloadset
    Returns:
    initializedPayloadset with given Data
  - Payloadset prepareAndValidatePayloads(Payloadset payloadset)
    
    This method is responsible for validating the payloads prepared to be sent to the CloudBees CD server
    Parameters:
    payloadset - Payloadset that was formed by the buildPayloadSet(FlowPlugin, Dataset)
    Returns:
    samePayloadset with validated fields.
  - Payload prepareAndValidateSinglePayload(Payload payload, Engine reportingEngine)
    
    This method compares the payload with format defined on CloudBees CD server
    Parameters:
    payload - Payload that will be validated
    reportingEngine - Engine instance that will get the definition from the CloudBees server
    Returns:
    samePayload with validated fields.
  - Dataset rawRecordsToDataset(java.util.List<java.util.Map<java.lang.String, java.lang.Object>> records)
    
    Converts objects returned from the reporting system to a Data
    Parameters:
    records - List of records returned from the reporting system
    Returns:
    Dataset built from the given records
  - static java.lang.Object validateAndConvertRow(java.lang.String fieldName, java.lang.Object rowValue, java.util.Map<java.lang.String, java.lang.Object> fieldDefinition)
    
    This method validates that the value for given key corresponds with the type that is defined on the CloudBees server.
    Parameters:
    fieldName - name of the payload key
    rowValue - value for the payload key
    fieldDefinition - Map returned by CloudBees CD API getReportObjectAttributes() method
    Returns:
    validated value

Summary:
Property
Method

| Detail:
Property
Method

Groovy Documentation

[Groovy] Class Reporting

Component state

Terminology

DevOps Insight Center

Release Command Center

Data source

Report Object Type

Payload

Records set

Data set

Payload set

Transform script

Metadata

Data Flow

What do you need to implement reporting?

An unique key/metric/value of each record/payload.

An API that allows you to get latest record.

How to implement reporting?

Default assumptions

Metadata compare

Transforming report object into the payload.

Properties Summary

Methods Summary

Inherited Methods Summary

Property Detail

Metadata currentMetadata

static FWLog fwLog

int initialRecordsCount

boolean isPreviewMode

MetadataFactory metadataFactory

java.lang.String metadataUniqueKey

java.util.ArrayList<java.lang.String> payloadKeys

java.lang.String pluginName

FlowPlugin pluginObject

java.util.ArrayList<java.lang.String> reportObjectTypes

Engine reportingEngine

Transformer transformer

Method Detail

Dataset buildDataset(FlowPlugin plugin, java.util.List<java.util.Map> records)

protected java.lang.String buildMetadataLocation()

Payloadset buildPayloadSet(FlowPlugin plugin, Dataset records)

void collectReportingData()

int compareMetadata(@org.jetbrains.annotations.NotNull Metadata currentMetadata, @org.jetbrains.annotations.NotNull Metadata newMetadata)

@org.jetbrains.annotations.Nullable java.util.Map<java.lang.String, java.lang.Object> getLastRecord(FlowPlugin plugin)

@org.jetbrains.annotations.Nullable java.util.List<java.util.Map<java.lang.String, java.lang.Object>> getRecordsAfter(FlowPlugin plugin, Metadata metadata)

Component init(@org.jetbrains.annotations.NotNull java.util.Map<java.lang.String, java.lang.Object> configuration, @org.jetbrains.annotations.Nullable FlowPlugin plugin)

@org.jetbrains.annotations.Nullable /** If a parameter named 'transformScript' was found and contains a closure an instance of the {@link Transformer} * will be initialized to process report objects. */ protected Transformer initTransformer(java.lang.String transformScriptValue)

@org.jetbrains.annotations.Nullable java.util.List<java.util.Map<java.lang.String, java.lang.Object>> initialGetRecords(FlowPlugin plugin, int initialRecordsCount)

boolean isEFComponent()

boolean isPreview()

Dataset newDataset(@org.jetbrains.annotations.NotNull java.util.ArrayList<java.lang.String> reportObjectTypes, java.util.List<Data> data)

MetadataFactory newMetadataFactory()

Payloadset newPayloadset(java.util.ArrayList<java.lang.String> reportObjectTypes, java.util.ArrayList<Payload> payloads = [])

Payloadset prepareAndValidatePayloads(Payloadset payloadset)

Payload prepareAndValidateSinglePayload(Payload payload, Engine reportingEngine)

Dataset rawRecordsToDataset(java.util.List<java.util.Map<java.lang.String, java.lang.Object>> records)

static java.lang.Object validateAndConvertRow(java.lang.String fieldName, java.lang.Object rowValue, java.util.Map<java.lang.String, java.lang.Object> fieldDefinition)