DataStoryPatternLibrabry

Data Story Patterns Library is a repository with pattern analysis designated for Linked Open Statistical Data. Story Patterns were retrieved from literture reserach udenr general subject of "data journalism".

Installation

pip install datastories

Requirements will be automatically installed with package

###Import/Usage

import datastories.analytical as patterns

patterns.DataStoryPattern(sparqlendpointurl, jsonmetadata)

Object created allow to query SPARQL endpoint based on JSON meatadat provided.

JSON Template

{
    "cube_key" : {
		"title":"title of cube",
		"dataset_structure":"URI for cube structure",
        "dimensions":{
            "dimension_key":{
                "dimension_title":"Title of diemnsion",
                "dimension_url":"URI for dimension",
                "dimension_prefix":"URI for dimension's values"
            },
            "dimension_key":{
                "dimension_title":"Title of diemnsion",
                "dimension_url":"URI for dimension",
                "dimension_prefix":"URI for dimension's values"
            }
		},
		"hierarchical_dimensions":{
			"dimension_key":{
                "dimension_title":"Title of diemnsion",
                "dimension_url":"URI for dimension",
                "dimension_prefix":"URI for dimension's values",
				"dimension_levels":
				{
					"level_key":"integer(granularity level)",
					"level_key":"integer(granularity level)"

				}
			}
		},
		"measures":{
			"measure_key":{
			"measure_title":"Title of measure",
			"measure_url":"URI for measure"
			}

		}
    }
}

Patterns Description

• Measurement and Counting
• League Table
• Internal Comprison
• Profile Outliers
• Dissect Factors
• Highlight Contrast
• Start Big Drill Down
• Start Small Zoom Out
• Analysis By Category
• Explore Intersection
• Narrating Change Over Time

MCounting

Measurement and Counting Arithemtical operators applied to whole dataset - basic information regarding data

Attributes

def MCounting(self,cube="",dims=[],meas=[],hierdims=[],count_type="raw",df=pd.DataFrame() )

Output

Based on count_type value

LTable

LeagueTable - sorting and extraction specific amount of records

Attributes

def LTable(self,cube=[],dims=[],meas=[],hierdims=[], columns_to_order="", order_type="asc", number_of_records=20,df=pd.DataFrame())

Output

Based on sort_type value

InternalComparison

InternalComparison - comparison of numeric values related to textual values within one column

Attributes

def InternalComparison(self,cube="",dims=[],meas=[],hierdims=[],df=pd.DataFrame(), dim_to_compare="",meas_to_compare="",comp_type="")

Output

Independent from comp_type selected, output data will have additional column with numerical column meas_to_compare processed in specific way.

Available types of comparison comp_type

ProfileOutliers

ProfileOutliers - detection of unusual values within data (anomalies)

Attributes

def ProfileOutliers(self,cube=[],dims=[],meas=[],hierdims=[],df=pd.DataFrame(), displayType="outliers_only")

Output

Pattern analysis using python scipy library will perform quick exploration in serach of unusual values within data.

Based on display_type parameter data will be displayed with/without ddetected unusual values.

Available types of displaying display_type

DissectFactors

DissectFactors - decomposition of data based on values in dim_to_dissect

Attributes

def DissectFactors(self,cube="",dims=[],meas=[],hierdims=[],df=pd.DataFrame(),dim_to_dissect="")

Output

As an output, data will be decomposed in a form of a dictionary, where each subset have values only related to specific value. Dictionary of subdataset will be constructed as a series of paiers where key per each susbet will values from dim_to_dissect and this key value will be data, where yhis key value was occurring.

HighlightContrast

HighlightContrast - partial difference within values related to one textual column

Attributes

def HighlightContrast(self,cube="",dims=[],meas=[],hierdims=[],df=pd.DataFrame(),dim_to_contrast="",contrast_type="",meas_to_contrast="")

Output

Independent from contrast_type selected, output data will have additional column with numerical column meas_to_contrast processed in specific way.

Available types of comparison contrast_type

StartBigDrillDown

StartBigDrillDown - data retrieval from multiple hierachical levels.

This pattern can be only applied to data not stored already in DataFrame

Attributes

def StartBigDrillDown(self,cube="",dims=[],meas=[],hierdim_drill_down=[])

Output

As an output, data will be retrieved in a form of a dictionary, where each dataset will be retrieved from different hierachy level. List will be provided inhierdim_drill_down. Hierachy levels provided by in parameter will automatically sorted in order from most general to most detailed level based on metadata provided.

StartSmallZoomOut

StartSmallZoomOut - data retrieval from multiple hierachical levels.

This pattern can be only applied to data not stored already in DataFrame

Attributes

def StartSmallZoomOut(self,cube="",dims=[],meas=[],hierdim_zoom_out=[])

Output

As an output, data will be retrieved in a form of a dictionary, where each dataset will be retrieved from different hierachy level. List will be provided inhierdim_zoom_out. Hierachy levels provided by in parameter will automatically sorted in order from most detaile to most general level based on metadata provided.

AnalysisByCategory

AnalysisByCategory - ecomposition of data based on values in dim_for_category with analysis performed on each susbet

Attributes

def AnalysisByCategory(self,cube="",dims=[],meas=[],hierdims=[],df=pd.DataFrame(),dim_for_category="",meas_to_analyse="",analysis_type="min"):

Output

As an output, data will be decomposed in a form of a dictionary, where each subset have values only related to specific value. Such subset will get analysed based on analysis_type parameter

Available types of analysis analysis_type

ExploreIntersection

Attributes

def ExploreIntersection(self, dim_to_explore=""):

Output

Pattern will return series of datasets, where each will represent occurence of dim_to_explore in one cube

NarratingChangeOverTime

Attributes

def NarrChangeOT(self,cube="",dims=[],meas=[],hierdims=[],df=pd.DataFrame(),meas_to_narrate="",narr_type="")

Output

Independent from narr_type selected, output data will have additional column with numerical values processed in specific way.

Available types of analysis narr_type