dtale package¶
Subpackages¶
- dtale.charts package
- dtale.cli package
- dtale.dash_application package
- Subpackages
- Submodules
- dtale.dash_application.charts module
- dtale.dash_application.custom_geojson module
- dtale.dash_application.drilldown_modal module
- dtale.dash_application.extended_aggregations module
- dtale.dash_application.lock_zoom module
- dtale.dash_application.saved_charts module
- dtale.dash_application.topojson_injections module
- dtale.dash_application.utils module
- dtale.dash_application.views module
- Module contents
- dtale.django package
- dtale.translations package
Submodules¶
dtale.app module¶
-
class
dtale.app.
DtaleFlask
(import_name, reaper_on=True, url=None, app_root=None, *args, **kwargs)[source]¶ Bases:
flask.app.Flask
Overriding Flask’s implementation of get_send_file_max_age, test_client & run
Parameters: - import_name – the name of the application package
- reaper_on (bool) – whether to run auto-reaper subprocess
- args – Optional arguments to be passed to
flask:flask.Flask
- kwargs – Optional keyword arguments to be passed to
flask:flask.Flask
-
build_reaper
(timeout=3600.0)[source]¶ Builds D-Tale’s auto-reaping process to cleanup process after an hour of inactivity
Parameters: timeout (float) – time in seconds before D-Tale is shutdown for inactivity, defaults to one hour
-
get_send_file_max_age
(name)[source]¶ Overriding Flask’s implementation of get_send_file_max_age so we can lower the timeout for javascript and css files which are changed more often
Parameters: name – filename Returns: Flask’s default behavior for get_send_max_age if filename is not in SHORT_LIFE_PATHS otherwise SHORT_LIFE_TIMEOUT
-
run
(*args, **kwargs)[source]¶ Parameters: - args – Optional arguments to be passed to
flask:flask.run()
- kwargs – Optional keyword arguments to be passed to
flask:flask.run()
- args – Optional arguments to be passed to
-
class
dtale.app.
DtaleFlaskTesting
(*args, **kwargs)[source]¶ Bases:
flask.testing.FlaskClient
Overriding Flask’s implementation of flask.FlaskClient so we can control the port associated with tests.
This class is required for setting the port on your test so that we won’t have SETTING keys colliding with other tests since the default for every test would be 80.
Parameters: - args – Optional arguments to be passed to
flask:flask.FlaskClient
- kwargs – Optional keyword arguments to be passed to
flask:flask.FlaskClient
- args – Optional arguments to be passed to
-
dtale.app.
build_app
(url=None, reaper_on=True, app_root=None, additional_templates=None, **kwargs)[source]¶ Builds
flask:flask.Flask
application encapsulating endpoints for D-Tale’s front-endParameters: - url (str, optional) – optional parameter which sets the host & root for any internal endpoints (ex: pinging shutdown)
- reaper_on (bool) – whether to run auto-reaper subprocess
- app_root (str, optional) – Optional path to prepend to the routes of D-Tale. This is used when making use of Jupyterhub server proxy
- additional_templates – path(s) to any other jinja templates you would like to load. This comes into play if you’re embedding D-Tale into your own Flask app
Type: str, list, optional
Returns: flask:flask.Flask
applicationReturn type:
-
dtale.app.
find_free_port
()[source]¶ Searches for free port on executing server to run the
flask:flask.Flask
process. Checks ports in range specified using environment variables:DTALE_MIN_PORT (default: 40000) DTALE_MAX_PORT (default: 49000)
The range limitation is required for usage in tools such as jupyterhub. Will raise an exception if an open port cannot be found.
Returns: port number Return type: int
-
dtale.app.
get_instance
(data_id)[source]¶ Returns a
dtale.views.DtaleData
object for the data_id passed as input, will return None if the data_id does not existParameters: data_id (int) – integer identifier for a D-Tale process’s data Returns: dtale.views.DtaleData
-
dtale.app.
initialize_process_props
(host=None, port=None, force=False)[source]¶ Helper function to initalize global state corresponding to the host & port being used for your
flask:flask.Flask
processParameters: - host (str, optional) – hostname to use otherwise it will default to the output of
socket.gethostname()
- port (str, optional) – port to use otherwise default to the output of
dtale.app.find_free_port()
- force (bool) – boolean flag to determine whether to ignore the
dtale.app.find_free_port()
function
Returns: - host (str, optional) – hostname to use otherwise it will default to the output of
-
dtale.app.
offline_chart
(df, chart_type=None, query=None, x=None, y=None, z=None, group=None, agg=None, window=None, rolling_comp=None, barmode=None, barsort=None, yaxis=None, filepath=None, title=None, **kwargs)[source]¶ Builds the HTML for a plotly chart figure to saved to a file or output to a jupyter notebook
Parameters: - df (
pandas.DataFrame
) – integer string identifier for a D-Tale process’s data - chart_type (str) – type of chart, possible options are line|bar|pie|scatter|3d_scatter|surface|heatmap
- query (str, optional) – pandas dataframe query string
- x (str) – column to use for the X-Axis
- y (list of str) – columns to use for the Y-Axes
- z (str, optional) – column to use for the Z-Axis
- group (list of str or str, optional) – column(s) to use for grouping
- agg (str, optional) – specific aggregation that can be applied to y or z axes. Possible values are: count, first, last mean, median, min, max, std, var, mad, prod, sum. This is included in label of axis it is being applied to.
- window (int, optional) – number of days to include in rolling aggregations
- rolling_comp (str, optional) – computation to use in rolling aggregations
- barmode (str, optional) – mode to use for bar chart display. possible values are stack|group(default)|overlay|relative
- barsort (str, optional) – axis name to sort the bars in a bar chart by (default is the ‘x’, but other options are any of columns names used in the ‘y’ parameter
- filepath (str, optional) – location to save HTML output
- title (str, optional) – Title of your chart
- kwargs (dict) – optional keyword arguments, here in case invalid arguments are passed to this function
Returns: possible outcomes are: - if run within a jupyter notebook and no ‘filepath’ is specified it will print the resulting HTML
within a cell in your notebook
- if ‘filepath’ is specified it will save the chart to the path specified
- otherwise it will return the HTML output as a string
- df (
-
dtale.app.
show
(data=None, data_loader=None, name=None, context_vars=None, **options)[source]¶ Entry point for kicking off D-Tale
flask:flask.Flask
process from python processParameters: - data (
pandas.DataFrame
orpandas.Series
orpandas.DatetimeIndex
orpandas.MultiIndex
, optional) – data which D-Tale will display - host (str, optional) – hostname of D-Tale, defaults to 0.0.0.0
- port (str, optional) – port number of D-Tale process, defaults to any open port on server
- name (str, optional) – optional label to assign a D-Tale process
- debug (bool, optional) – will turn on
flask:flask.Flask
debug functionality, defaults to False - subprocess (bool, optional) – run D-Tale as a subprocess of your current process, defaults to True
- data_loader (func, optional) – function to load your data
- reaper_on (bool, optional) – turn on subprocess which will terminate D-Tale after 1 hour of inactivity
- open_browser (bool, optional) – if true, this will try using the
webbrowser
package to automatically open your default browser to your D-Tale process - notebook (bool, optional) – if true, this will try displaying an
IPython.display.IFrame
- force (bool, optional) – if true, this will force the D-Tale instance to run on the specified host/port by killing any other process running at that location
- context_vars (dict, optional) – a dictionary of the variables that will be available for use in user-defined expressions, such as filters
- ignore_duplicate (bool, optional) – if true, this will not check if this data matches any other data previously loaded to D-Tale
- app_root (str, optional) – Optional path to prepend to the routes of D-Tale. This is used when making use of Jupyterhub server proxy
- allow_cell_edits (bool, optional) – If false, this will not allow users to edit cells directly in their D-Tale grid
- inplace (bool, optional) – If true, this will call reset_index(inplace=True) on the dataframe used as a way to save memory. Otherwise this will create a brand new dataframe, thus doubling memory but leaving the dataframe input unchanged.
- drop_index (bool, optional) – If true, this will drop any pre-existing index on the dataframe input.
- hide_shutdown (bool, optional) – If true, this will hide the “Shutdown” buton from users
- github_fork (bool, optional) – If true, this will display a “Fork me on GitHub” ribbon in the upper right-hand corner of the app
- hide_drop_rows (bool, optional) – If true, this will hide the “Drop Rows” buton from users
Example: >>> import dtale >>> import pandas as pd >>> df = pandas.DataFrame([dict(a=1,b=2,c=3)]) >>> dtale.show(df) D-Tale started at: http://hostname:port
..link displayed in logging can be copied and pasted into any browser
- data (
dtale.auth module¶
dtale.code_export module¶
dtale.column_analysis module¶
dtale.column_builders module¶
dtale.column_filters module¶
dtale.column_replacements module¶
-
class
dtale.column_replacements.
ColumnReplacement
(data_id, col, replacement_type, cfg, name=None)[source]¶ Bases:
object
dtale.combine_data module¶
dtale.config module¶
dtale.correlations module¶
dtale.data_reshapers module¶
dtale.datasets module¶
dtale.describe module¶
-
dtale.describe.
load_describe
(column_series, additional_aggs=None)[source]¶ Helper function for grabbing the output from
pandas.Series.describe()
in a JSON serializable formatParameters: column_series ( pandas.Series
) – data to describeReturns: JSON serializable dictionary of the output from calling pandas.Series.describe()
dtale.duplicate_checks module¶
-
exception
dtale.duplicate_checks.
NoDuplicatesException
[source]¶ Bases:
ValueError
Container class for any instance where a user tries to remove duplicates when they don’t exist.
-
exception
dtale.duplicate_checks.
NoDuplicatesToShowException
[source]¶ Bases:
ValueError
Container class for any instance where a user tries to show duplicates that don’t exist.
-
exception
dtale.duplicate_checks.
RemoveAllDataException
[source]¶ Bases:
ValueError
Container class for any instance where a user tries remove duplicates and it returns an empty dataframe.
dtale.env_util module¶
-
dtale.env_util.
open_browser
(url)[source]¶ Open a web browser pointing to a given URL.
We use this function instead of Python’s webbrowser module because this way we can capture stdout/stderr to avoid polluting the terminal with the browser’s messages. For example, Chrome always prints things like “Created new window in existing browser session”, and those get on the user’s way.
- url : str
- The URL. Must include the protocol.
dtale.gage_rnr module¶
-
class
dtale.gage_rnr.
Component
[source]¶ Bases:
enum.Enum
Enum containing the different Variance parts of GageRnR.
-
MEASUREMENT
= 3¶
-
OPERATOR
= 0¶
-
OPERATOR_BY_PART
= 2¶
-
PART
= 1¶
-
TOTAL
= 4¶
-
-
class
dtale.gage_rnr.
GageRnR
(data)[source]¶ Bases:
dtale.gage_rnr.Statistics
Main class for calculating GageRnR.
-
GRR
= 'GageRnR'¶
-
title
= 'Gage R&R'¶
-
dtale.global_state module¶
-
class
dtale.global_state.
DtaleInstance
(data)[source]¶ Bases:
object
-
context_variables
¶
-
data
¶
-
dataset
¶
-
dataset_dim
¶
-
dtypes
¶
-
history
¶
-
is_xarray_dataset
¶
-
metadata
¶
-
name
¶
-
settings
¶
-
-
dtale.global_state.
use_redis_store
(directory, *args, **kwargs)[source]¶ Configure dtale to use redis for the global data store. Useful for web servers.
Parameters: - db_folder (str) – folder that db files will be stored in
- args – All other arguments supported by the redislite.Redis() class
- kwargs – All other keyword arguments supported by the redislite.Redis() class
Returns: None
-
dtale.global_state.
use_shelve_store
(directory)[source]¶ Configure dtale to use python’s standard ‘shelve’ library for a persistent global data store.
Parameters: directory (str) – directory that the shelve db files will be stored in Returns: None
-
dtale.global_state.
use_store
(store_class, create_store)[source]¶ Customize how dtale stores and retrieves global data. By default it uses global dictionaries, but this can be problematic if there are memory limitations or multiple python processes are running. Ex: a web server with multiple workers (processes) for processing requests.
Parameters: - store_class – Class providing an interface to the data store. To be valid, it must: 1. Implement get, keys, items clear, __setitem__, __delitem__, __iter__, __len__, __contains__. 2. Either be a subclass of MutableMapping or implement the ‘to_dict’ method.
- create_store – Factory function for producing instances of <store_class>. Must take ‘name’ as the only parameter.
Returns: None
dtale.predefined_filters module¶
dtale.query module¶
-
dtale.query.
run_query
(df, query, context_vars=None, ignore_empty=False, pct=100, pct_type='random')[source]¶ Utility function for running
pandas.DataFrame.query
. This function contains extra logic to handle when column names contain special characters. Looks like pandas will be handling this in a future version: https://github.com/pandas-dev/pandas/issues/27017The logic to handle these special characters in the meantime is only available in Python 3+
Parameters: - df (
pandas.DataFrame
) – input dataframe - query (str) – query string
- context_vars (dict, optional) – dictionary of user-defined variables which can be referenced by name in query strings
- pct (int, optional) – random percentage of dataframe to load
Returns: filtered dataframe
- df (
dtale.timeseries_analysis module¶
dtale.utils module¶
-
exception
dtale.utils.
ChartBuildingError
(error, details=None)[source]¶ Bases:
Exception
Exception for signalling there was an issue constructing the data for your chart.
-
exception
dtale.utils.
DuplicateDataError
(data_id)[source]¶ Bases:
Exception
Exception for signalling that similar data is trying to be loaded to D-Tale again. Is this correct?
-
class
dtale.utils.
JSONFormatter
(nan_display='', as_string=False)[source]¶ Bases:
object
Class for formatting dictionaries and lists of dictionaries into JSON compliant data
Example: >>> nan_display = 'nan' >>> f = JSONFormatter(nan_display) >>> f.add_int(1, 'a') >>> f.add_float(2, 'b') >>> f.add_string(3, 'c') >>> jsonify(f.format_dicts([dict(a=1, b=2.0, c='c')]))
-
dtale.utils.
build_formatters
(df, nan_display=None)[source]¶ Helper around
dtale.utils.grid_formatters()
that will build a formatter for the data being fed into a chart as well as a formatter for the min/max values for each column used in the chart data.Parameters: df ( pandas.DataFrame
) – dataframe which contains column names and data types for formattersReturns: json formatters for chart data and min/max values for each column used in the chart Return type: ( dtale.utils.JSONFormatter
,dtale.utils.JSONFormatter
)
-
dtale.utils.
build_shutdown_url
(base)[source]¶ Builds the shutdown endpoint for the specified port
Parameters: port (str) – integer string for a D-Tale process’s port Returns: URL string of the shutdown endpoint for the current server and port passed
-
dtale.utils.
build_url
(port, host)[source]¶ Returns full url combining host(if not specified will use the output of
socket.gethostname()
) & portParameters: Returns: str
-
dtale.utils.
classify_type
(type_name)[source]¶ Parameters: type_name – string label for value from pandas.DataFrame.dtypes
Returns: shortened string label for dtype S = str B = bool F = float I = int D = timestamp or datetime TD = timedelta Return type: str
-
dtale.utils.
dict_merge
(d1, d2, *args)[source]¶ Merges two dictionaries. Items of the second dictionary will replace items of the first dictionary if there are any overlaps. Either dictionary can be None. An empty dictionary {} will be returned if both dictionaries are None.
Parameters: - d1 (dict) – First dictionary can be None
- d2 – Second dictionary can be None
Returns: new dictionary with the contents of d2 overlaying the contents of d1
Return type:
-
dtale.utils.
find_dtype
(s)[source]¶ Helper function to determine the dtype of a
pandas.Series
-
dtale.utils.
find_selected_column
(data, col)[source]¶ In case we come across a series which after reset_index() it’s columns are [date, security_id, values] in which case we want the last column
Parameters: - data (
pandas.DataFrame
) – dataframe - col (str) – column name
Returns: column name if it exists within the dataframe’s columns, the last column within the dataframe otherwise
Return type: - data (
-
dtale.utils.
flatten_lists
(lists)[source]¶ - Take an iterable containing iterables and flatten them into one list.
- [[1], [2], [3, 4]] => [1, 2, 3, 4]
-
dtale.utils.
format_grid
(df, overrides=None)[source]¶ Translate
pandas.DataFrame
to well-formed JSON. Structure is as follows: {- results: [
- {col1: val1_row1,…,colN: valN_row1}, …, {col1: val1_rowN,…,colN: valN_rowN},
], columns: [
{name: col1, dtype: int}, …, {name: colN, dtype: float},]
}
Parameters: df ( pandas.DataFrame
) – dataframeReturns: JSON
-
dtale.utils.
get_bool_arg
(r, name)[source]¶ Retrieve argument from
flask:flask.request
and convert to booleanParameters: - r –
flask:flask.request
- name – argument name
Type: str
Returns: True if lowercase value equals ‘true’, False otherwise
- r –
-
dtale.utils.
get_dtypes
(df)[source]¶ Build dictionary of column/dtype name pairs from
pandas.DataFrame
-
dtale.utils.
get_float_arg
(r, name, default=None)[source]¶ Retrieve argument from
flask:flask.request
and convert to floatParameters: - r –
flask:flask.request
- name – argument name
- default – default value if parameter is non-existent, defaults to None
Type: str
Returns: float argument value
- r –
-
dtale.utils.
get_host
(host=None)[source]¶ Returns host input if it exists otherwise the output of
socket.gethostname()
Parameters: host (str, optional) – hostname, can start with ‘http://’, ‘https://’ or just the hostname itself Returns: str
-
dtale.utils.
get_int_arg
(r, name, default=None)[source]¶ Retrieve argument from
flask:flask.request
and convert to integerParameters: - r –
flask:flask.request
- name – argument name
- default – default value if parameter is non-existent, defaults to None
Type: str
Returns: integer argument value
- r –
-
dtale.utils.
get_json_arg
(r, name, default=None)[source]¶ Retrieve argument from
flask:flask.request
and parse JSON to python data structureParameters: - r –
flask:flask.request
- name – argument name
- default – default value if parameter is non-existent, defaults to None
Type: str
Returns: parsed JSON
- r –
-
dtale.utils.
get_str_arg
(r, name, default=None)[source]¶ Retrieve argument from
flask:flask.request
and convert to stringParameters: - r –
flask:flask.request
- name – argument name
- default – default value if parameter is non-existent, defaults to None
Type: str
Returns: string argument value
- r –
-
dtale.utils.
grid_columns
(df)[source]¶ Build list of {name, dtype} dictionaries for columns in
pandas.DataFrame
-
dtale.utils.
grid_formatter
(col_types, nan_display='', overrides=None, as_string=False)[source]¶ Build
dtale.utils.JSONFormatter
frompandas.DataFrame
-
dtale.utils.
json_date
(x, fmt='%Y-%m-%d %H:%M:%S.%f', nan_display='', **kwargs)[source]¶ Convert value to date string to be used within JSON output
Parameters: - x – value to be converted to date string
- fmt – the data string formatting to be applied
- nan_display – if x is
numpy.nan
then return this value
Returns: date string value
Return type: str (YYYY-MM-DD)
-
dtale.utils.
json_float
(x, precision=2, nan_display='nan', inf_display='inf', as_string=False)[source]¶ Convert value to float to be used within JSON output
Parameters: - x – value to be converted to integer
- precision – precision of float to be returned
- nan_display – if x is
numpy.nan
then return this value - inf_display – if x is
numpy.inf
then return this value - as_string – return float as a formatted string (EX: 1,234.5643)
Returns: float value
Return type:
-
dtale.utils.
json_int
(x, nan_display='', as_string=False, fmt='{:, d}')[source]¶ Convert value to integer to be used within JSON output
Parameters: - x – value to be converted to integer
- nan_display – if x is
numpy.nan
then return this value - as_string – return integer as a formatted string (EX: 1,000,000)
Returns: integer value
Return type:
-
dtale.utils.
json_string
(x, nan_display='', **kwargs)[source]¶ convert value to string to be used within JSON output
If a
python.UnicodeEncodeError
occurs thenstr.encode
will be called on inputParameters: - x – value to be converted to string
- nan_display – if x is
numpy.nan
then return this value
Returns: string value
Return type:
-
dtale.utils.
json_timestamp
(x, nan_display='', **kwargs)[source]¶ Convert value to timestamp (milliseconds) to be used within JSON output :param x: value to be converted to milliseconds :param nan_display: if x is
numpy.nan
then return this value :return: millisecond value :rtype: bigint
-
dtale.utils.
jsonify
(return_data={}, **kwargs)[source]¶ Overriding Flask’s jsonify method to account for extra error handling
Parameters: - return_data – dictionary of data to be passed to
flask:flask.jsonify()
- kwargs – Optional keyword arguments merged into return_data
Returns: output of
flask:flask.jsonify()
- return_data – dictionary of data to be passed to
-
dtale.utils.
retrieve_grid_params
(req, props=None)[source]¶ Pull out grid parameters from
flask:flask.request
arguments and return as a dictParameters: - req –
flask:flask.request
- props (list) – argument names
Returns: dictionary of argument/value pairs
Return type: - req –
-
dtale.utils.
running_with_flask_debug
()[source]¶ Checks to see if D-Tale has been initiated from Flask
Returns: True if executed from test, False otherwise Return type: bool
-
dtale.utils.
running_with_pytest
()[source]¶ Checks to see if D-Tale has been initiated from test
Returns: True if executed from test, False otherwise Return type: bool
-
dtale.utils.
sort_df_for_grid
(df, params)[source]¶ Sort dataframe based on ‘sort’ property in parameter dictionary. Sort configuration is of the following shape: {
- sort: [
- [col1, ASC], [col2, DESC], …
]
}
Parameters: - df (
pandas.DataFrame
) – dataframe - params (dict) – arguments from
flask:flask.request
Returns: sorted dataframe
Return type:
dtale.views module¶
-
class
dtale.views.
DtaleData
(data_id, url, is_proxy=False, app_root=None)[source]¶ Bases:
object
Wrapper class to abstract the global state of a D-Tale process while allowing a user to programatically interact with a running D-Tale instance
Parameters: - Attributes:
- _data_id data identifier
_url
flask:flask.Flask
endpoint _notebook_handle reference to the most recentIPython.display.DisplayHandle
created
Example: >>> import dtale >>> import pandas as pd >>> df = pd.DataFrame([dict(a=1,b=2,c=3)]) >>> d = dtale.show(df) >>> tmp = d.data.copy() >>> tmp['d'] = 4 >>> d.data = tmp >>> d.kill()
-
adjust_cell_dimensions
(width='100%', height=350)[source]¶ If you are running ipython>=5.0 then this will update the most recent notebook cell you displayed D-Tale in for this instance with the height/width properties you have passed in as input
Parameters: - width – width of the ipython cell
- height – height of the ipython cell
-
cleanup
()[source]¶ Helper function to clean up data associated with this instance from global state.
-
data
¶ Property which is a reference to the globally stored data associated with this instance
-
is_up
()[source]¶ Helper function to pass instance’s endpoint to
dtale.views.is_up()
-
kill
()[source]¶ Helper function to pass instance’s endpoint to
dtale.views.kill()
-
main_url
()[source]¶ Helper function creating main
flask:flask.Flask
route using instance’s url & data_id :return: str
-
notebook
(route='/dtale/iframe/', params=None, width='100%', height=475)[source]¶ Helper function which checks to see if
flask:flask.Flask
process is up and running and then tries to build anIPython.display.IFrame
and runIPython.display.display
on it so it will be displayed in the ipython notebook which invoked it.A reference to the
IPython.display.DisplayHandle
is stored in _notebook_handle for updating if you are running ipython>=5.0Parameters:
-
notebook_charts
(chart_type='line', query=None, x=None, y=None, z=None, group=None, agg=None, window=None, rolling_comp=None, barmode=None, barsort=None, width='100%', height=800)[source]¶ Helper function to build an ipython:IPython.display.IFrame pointing at the charts popup
Parameters: - chart_type (str) – type of chart, possible options are line|bar|pie|scatter|3d_scatter|surface|heatmap
- query (str, optional) – pandas dataframe query string
- x (str) – column to use for the X-Axis
- y (list of str) – columns to use for the Y-Axes
- z (str, optional) – column to use for the Z-Axis
- group (list of str or str, optional) – column(s) to use for grouping
- agg (str, optional) – specific aggregation that can be applied to y or z axes. Possible values are: count, first, last, mean, median, min, max, std, var, mad, prod, sum. This is included in label of axis it is being applied to.
- window (int, optional) – number of days to include in rolling aggregations
- rolling_comp (str, optional) – computation to use in rolling aggregations
- barmode (str, optional) – mode to use for bar chart display. possible values are stack|group(default)|overlay|relative
- barsort (str, optional) – axis name to sort the bars in a bar chart by (default is the ‘x’, but other options are any of columns names used in the ‘y’ parameter
- width (str or int, optional) – width of the ipython cell
- height (str or int, optional) – height of the ipython cell
Returns:
-
notebook_correlations
(col1, col2, width='100%', height=475)[source]¶ Helper function to build an ipython:IPython.display.IFrame pointing at the correlations popup
Parameters: Returns:
-
offline_chart
(chart_type=None, query=None, x=None, y=None, z=None, group=None, agg=None, window=None, rolling_comp=None, barmode=None, barsort=None, yaxis=None, filepath=None, title=None, **kwargs)[source]¶ Builds the HTML for a plotly chart figure to saved to a file or output to a jupyter notebook
Parameters: - chart_type (str) – type of chart, possible options are line|bar|pie|scatter|3d_scatter|surface|heatmap
- query (str, optional) – pandas dataframe query string
- x (str) – column to use for the X-Axis
- y (list of str) – columns to use for the Y-Axes
- z (str, optional) – column to use for the Z-Axis
- group (list of str or str, optional) – column(s) to use for grouping
- agg (str, optional) – specific aggregation that can be applied to y or z axes. Possible values are: count, first, last, mean, median, min, max, std, var, mad, prod, sum. This is included in label of axis it is being applied to.
- window (int, optional) – number of days to include in rolling aggregations
- rolling_comp (str, optional) – computation to use in rolling aggregations
- barmode (str, optional) – mode to use for bar chart display. possible values are stack|group(default)|overlay|relative
- barsort (str, optional) – axis name to sort the bars in a bar chart by (default is the ‘x’, but other options are any of columns names used in the ‘y’ parameter
- yaxis (dict, optional) – dictionary specifying the min/max for each y-axis in your chart
- filepath (str, optional) – location to save HTML output
- title (str, optional) – Title of your chart
- kwargs (dict) – optional keyword arguments, here in case invalid arguments are passed to this function
Returns: possible outcomes are: - if run within a jupyter notebook and no ‘filepath’ is specified it will print the resulting HTML
within a cell in your notebook
- if ‘filepath’ is specified it will save the chart to the path specified
- otherwise it will return the HTML output as a string
-
open_browser
()[source]¶ This function uses the
webbrowser
library to try and automatically open server’s default browser to this D-Tale instance
-
update_id
(new_data_id)[source]¶ Update current data_id to new data_id
Parameters: new_data_id – the data_id to update to
-
update_settings
(**updates)[source]¶ Helper function for updating instance-specific settings. For example: * allow_cell_edits - whether cells can be edited * locked - which columns are locked to the left of the grid * custom_formats - display formatting for specific columns * background_mode - different background displays in grid * range_highlights - specify background colors for ranges of values in the grid * vertical_headers - if True, then rotate column headers vertically
-
exception
dtale.views.
NoDataLoadedException
[source]¶ Bases:
Exception
Container class for any scenario where no data has been loaded into D-Tale.
This will usually force the user to load data using the CSV/TSV loader UI.
-
dtale.views.
base_render_template
(template, data_id, **kwargs)[source]¶ - Overriden version of Flask.render_template which will also include vital instance information
- settings
- version
- processes
-
dtale.views.
build_column
(data_id)[source]¶ flask:flask.Flask
route to handle the building of new columns in a dataframe. Some of the operations the are available are:- numeric: sum/difference/multiply/divide any combination of two columns or static values
- datetime: retrieving date properties (hour, minute, month, year…) or conversions of dates (month start, month
- end, quarter start…)
- bins: bucketing numeric data into bins using
pandas.cut
&pandas.qcut
Parameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- name – string from flask.request.args[‘name’] of new column to create
- type – string from flask.request.args[‘type’] of the type of column to build (numeric/datetime/bins)
- cfg – dict from flask.request.args[‘cfg’] of how to calculate the new column
Returns: JSON {success: True/False}
-
dtale.views.
build_context_variables
(data_id, new_context_vars=None)[source]¶ Build and return the dictionary of context variables associated with a process. If the names of any new variables are not formatted properly, an exception will be raised. New variables will overwrite the values of existing variables if they share the same name.
Parameters: Returns: dict of the context variables for this process
Return type:
-
dtale.views.
build_dtypes_state
(data, prev_state=None, ranges=None)[source]¶ Helper function to build globally managed state pertaining to a D-Tale instances columns & data types
Parameters: data ( pandas.DataFrame
) – dataframe to build data type information forReturns: a list of dictionaries containing column names, indexes and data types
-
dtale.views.
build_replacement
(data_id)[source]¶ flask:flask.Flask
route to handle the replacement of specific values within a column in a dataframe. Some of the operations the are available are:- spaces: replace values consisting of only spaces with a specific value
- value: replace specific values with a specific value or aggregation
- strings: replace values which contain a specific character or string (case-insensitive or not) with a
- specific value
- imputer: replace nan values using sklearn imputers iterative, knn or simple
Parameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- col – string from flask.request.args[‘col’] of the column to perform replacements upon
- type – string from flask.request.args[‘type’] of the type of replacement to perform (spaces/fillna/strings/imputer)
- cfg – dict from flask.request.args[‘cfg’] of how to calculate the replacements
Returns: JSON {success: True/False}
-
dtale.views.
check_duplicate_data
(data)[source]¶ This function will do a rough check to see if a user has already loaded this piece of data to D-Tale to avoid duplicated state. The checks that take place are:
- shape (# of rows & # of columns
- column names and ordering of columns (eventually might add dtype checking as well…)
Parameters: data ( pandas.DataFrame
) – dataframe to validate:raises
dtale.utils.DuplicateDataError
: if duplicate data exists
-
dtale.views.
describe
(data_id)[source]¶ flask:flask.Flask
route which returns standard details about column data usingpandas.DataFrame.describe()
to the front-end as JSONParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- column – required dash separated string “START-END” stating a range of row indexes to be returned to the screen
Returns: JSON { describe: object representing output from
pandas.Series.describe()
, unique_data: array of unique values when data has <= 100 unique values success: True/False}
-
dtale.views.
dtype_formatter
(data, dtypes, data_ranges, prev_dtypes=None)[source]¶ Helper function to build formatter for the descriptive information about each column in the dataframe you are viewing in D-Tale. This data is later returned to the browser to help with controlling inputs to functions which are heavily tied to specific data types.
Parameters: - data (
pandas.DataFrame
) – dataframe - dtypes (dict) – column data type
- data_ranges (dict, optional) – dictionary containing minimum and maximum value for column (if applicable)
- prev_dtypes (dict, optional) – previous column information for syncing updates to pre-existing columns
Returns: formatter function which takes column indexes and names
Return type: func
- data (
-
dtale.views.
dtypes
(data_id)[source]¶ flask:flask.Flask
route which returns a list of column names and dtypes to the front-end as JSONParameters: data_id (str) – integer string identifier for a D-Tale process’s data Returns: JSON { dtypes: [ {index: 1, name: col1, dtype: int64}, …, {index: N, name: colN, dtype: float64}], success: True/False
}
-
dtale.views.
format_data
(data, inplace=False, drop_index=False)[source]¶ - Helper function to build globally managed state pertaining to a D-Tale instances data. Some updates being made:
- convert all column names to strings
- drop any indexes back into the dataframe so what we are left is a natural index [0,1,2,…,n]
- convert inputs that are indexes into dataframes
- replace any periods in column names with underscores
Parameters: - data (
pandas.DataFrame
) – dataframe to build data type information for - allow_cell_edits (bool, optional) – If false, this will not allow users to edit cells directly in their D-Tale grid
- inplace (bool, optional) – If true, this will call reset_index(inplace=True) on the dataframe used as a way to save memory. Otherwise this will create a brand new dataframe, thus doubling memory but leaving the dataframe input unchanged.
- drop_index (bool, optional) – If true, this will drop any pre-existing index on the dataframe input.
Returns: formatted
pandas.DataFrame
and a list of strings constituting what columns were originally in the indexRaises: Exception if the dataframe contains two columns of the same name
-
dtale.views.
get_chart_data
(data_id)[source]¶ flask:flask.Flask
route which builds data associated with a chart.js chartParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- query – string from flask.request.args[‘query’] which is applied to DATA using the query() function
- x – string from flask.request.args[‘x’] column to be used as x-axis of chart
- y – string from flask.request.args[‘y’] column to be used as y-axis of chart
- group – string from flask.request.args[‘group’] comma-separated string of columns to group chart data by
- agg – string from flask.request.args[‘agg’] points to a specific function that can be applied to :func: pandas.core.groupby.DataFrameGroupBy. Possible values are: count, first, last mean, median, min, max, std, var, mad, prod, sum
Returns: JSON { data: {
series1: { x: [x1, x2, …, xN], y: [y1, y2, …, yN] }, series2: { x: [x1, x2, …, xN], y: [y1, y2, …, yN] }, …, seriesN: { x: [x1, x2, …, xN], y: [y1, y2, …, yN] },
}, min: minY, max: maxY,
} or {error: ‘Exception message’, traceback: ‘Exception stacktrace’}
-
dtale.views.
get_column_analysis
(data_id)[source]¶ flask:flask.Flask
route which returns output from numpy.histogram/pd.value_counts to front-end as JSONParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- col – string from flask.request.args[‘col’] containing name of a column in your dataframe
- type – string from flask.request.args[‘type’] to signify either a histogram or value counts
- query – string from flask.request.args[‘query’] which is applied to DATA using the query() function
- bins – the number of bins to display in your histogram, options on the front-end are 5, 10, 20, 50
- top – the number of top values to display in your value counts, default is 100
Returns: JSON {results: DATA, desc: output from pd.DataFrame[col].describe(), success: True/False}
-
dtale.views.
get_correlations
(data_id)[source]¶ flask:flask.Flask
route which gathers Pearson correlations against all combinations of columns with numeric data usingpandas.DataFrame.corr()
On large datasets with no
numpy.nan
data this code will usenumpy.corrcoef
for speed purposesParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- query – string from flask.request.args[‘query’] which is applied to DATA using the query() function
Returns: JSON { data: [{column: col1, col1: 1.0, col2: 0.99, colN: 0.45},…,{column: colN, col1: 0.34, col2: 0.88, colN: 1.0}],
} or {error: ‘Exception message’, traceback: ‘Exception stacktrace’}
-
dtale.views.
get_correlations_ts
(data_id)[source]¶ flask:flask.Flask
route which returns timeseries of Pearson correlations of two columns with numeric data usingpandas.DataFrame.corr()
Parameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- cols – comma-separated string from flask.request.args[‘cols’] containing names of two columns in dataframe
- dateCol – string from flask.request.args[‘dateCol’] with name of date-type column in dateframe for timeseries
Returns: JSON { data: {:col1:col2: {data: [{corr: 0.99, date: ‘YYYY-MM-DD’},…], max: 0.99, min: 0.99}
} or {error: ‘Exception message’, traceback: ‘Exception stacktrace’}
-
dtale.views.
get_data
(data_id)[source]¶ flask:flask.Flask
route which returns current rows from DATA (based on scrollbar specs and saved settings) to front-end as JSONParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- ids – required dash separated string “START-END” stating a range of row indexes to be returned to the screen
- query – string from flask.request.args[‘query’] which is applied to DATA using the query() function
- sort – JSON string from flask.request.args[‘sort’] which is applied to DATA using the sort_values() or sort_index() function. Here is the JSON structure: [col1,dir1],[col2,dir2],….[coln,dirn]
Returns: JSON { results: [
{dtale_index: 1, col1: val1_1, …,colN: valN_1}, …, {dtale_index: N2, col1: val1_N2, …,colN: valN_N2}
], columns: [{name: col1, dtype: ‘int64’},…,{name: colN, dtype: ‘datetime’}], total: N2, success: True/False
}
-
dtale.views.
get_filter_info
(data_id)[source]¶ flask:flask.Flask
route which returns a view-only version of the query, column filters & context variables to the front end.Parameters: data_id (str) – integer string identifier for a D-Tale process’s data Returns: JSON
-
dtale.views.
get_processes
()[source]¶ flask:flask.Flask
route which returns list of running D-Tale processes within current python processReturns: JSON { data: [ - {
- port: 1, name: ‘name1’, rows: 5, columns: 5, names: ‘col1,…,col5’, start: ‘2018-04-30 12:36:44’, ts: 1525106204000
}, …, {
port: N, name: ‘nameN’, rows: 5, columns: 5, names: ‘col1,…,col5’, start: ‘2018-04-30 12:36:44’, ts: 1525106204000}
], success: True/False
}
-
dtale.views.
get_scatter
(data_id)[source]¶ flask:flask.Flask
route which returns data used in correlation of two columns for scatter chartParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- cols – comma-separated string from flask.request.args[‘cols’] containing names of two columns in dataframe
- dateCol – string from flask.request.args[‘dateCol’] with name of date-type column in dateframe for timeseries
- date – string from flask.request.args[‘date’] date value in dateCol to filter dataframe to
Returns: JSON { data: [{col1: 0.123, col2: 0.123, index: 1},…,{col1: 0.123, col2: 0.123, index: N}], stats: { stats: {
correlated: 50, only_in_s0: 1, only_in_s1: 2, pearson: 0.987, spearman: 0.879,
} x: col1, y: col2
} or {error: ‘Exception message’, traceback: ‘Exception stacktrace’}
-
dtale.views.
handle_koalas
(data)[source]¶ Helper function to check if koalas is installed and also if incoming data is a koalas dataframe, if so convert it to
pandas.DataFrame
, otherwise simply return the original data structure.Parameters: data – data we want to check if its a koalas dataframe and if so convert to pandas.DataFrame
Returns: pandas.DataFrame
-
dtale.views.
in_ipython_frontend
()[source]¶ Helper function which is variation of
pandas.io.formats.console.in_ipython_frontend
which checks to see if we are inside an IPython zmq frontendReturns: True if D-Tale is being invoked within ipython notebook, False otherwise
-
dtale.views.
is_up
(base)[source]¶ This function checks to see if instance’s
flask:flask.Flask
process is up by hitting ‘health’ route.Using verify=False will allow us to validate instances being served up over SSL
Returns: True if flask:flask.Flask
process is up and running, False otherwise
-
dtale.views.
kill
(base)[source]¶ This function fires a request to this instance’s ‘shutdown’ route to kill it
-
dtale.views.
refresh_col_indexes
(data_id)[source]¶ Helper function to sync column indexes to current state of dataframe for data_id.
-
dtale.views.
startup
(url='', data=None, data_loader=None, name=None, data_id=None, context_vars=None, ignore_duplicate=True, allow_cell_edits=True, inplace=False, drop_index=False, precision=2, show_columns=None, hide_columns=None, optimize_dataframe=False, column_formats=None, nan_display=None, sort=None, locked=None, background_mode=None, range_highlights=None, app_root=None, is_proxy=None, vertical_headers=False)[source]¶ - Loads and stores data globally
- If data has indexes then it will lock save those columns as locked on the front-end
- If data has column named index it will be dropped so that it won’t collide with row numbering (dtale_index)
- Create location in memory for storing settings which can be manipulated from the front-end (sorts, filter, …)
Parameters: - url – the base URL that D-Tale is running from to be referenced in redirects to shutdown
- data –
pandas.DataFrame
orpandas.Series
- data_loader – function which returns
pandas.DataFrame
- name – string label to apply to your session
- data_id – integer id assigned to a piece of data viewable in D-Tale, if this is populated then it will override the data at that id
- context_vars (dict, optional) – a dictionary of the variables that will be available for use in user-defined expressions, such as filters
- ignore_duplicate – if set to True this will not test whether this data matches any previously loaded to D-Tale
- allow_cell_edits (bool, optional) – If false, this will not allow users to edit cells directly in their D-Tale grid
- inplace (bool, optional) – If true, this will call reset_index(inplace=True) on the dataframe used as a way to save memory. Otherwise this will create a brand new dataframe, thus doubling memory but leaving the dataframe input unchanged.
- drop_index (bool, optional) – If true, this will drop any pre-existing index on the dataframe input.
- precision (int, optional) – The default precision to display for float data in D-Tale grid
- show_columns (list, optional) – Columns to show on load, hide all others
- hide_columns (list, optional) – Columns to hide on load
- optimize_dataframe (boolean) – this will convert string columns with less certain then a certain number of distinct values into categories
- column_formats (dict, optional) – The formatting to apply to certain columns on the front-end
- sort (list[tuple], optional) – The sort to apply to the data on startup (EX: [(“col1”, “ASC”), (“col2”, “DESC”),…])
- locked (list, optional) – Columns to lock to the left of your grid on load
- background_mode (string, optional) –
Different background highlighting modes available on the frontend. Possible values are: - heatmap-all: turn on heatmap for all numeric columns where the colors are determined by
the range of values over all numeric columns combined- heatmap-col: turn on heatmap for all numeric columns where the colors are determined by
- the range of values in the column
- heatmap-col-[column name]: turn on heatmap highlighting for a specific column
- dtypes: highlight columns based on it’s data type
- missing: highlight any missing values (np.nan, empty strings, strings of all spaces)
- outliers: highlight any outliers
- range: highlight values for any matchers entered in the “range_highlights” option
- lowVariance: highlight values with a low variance
- range_highlights (dict, optional) – Definitions for equals, less-than or greater-than ranges for individual (or all) columns which apply different background colors to cells which fall in those ranges.
- vertical_headers (boolean, optional) – if True, then rotate column headers vertically
-
dtale.views.
test_filter
(data_id)[source]¶ flask:flask.Flask
route which will test out pandas query before it gets applied to DATA and return exception information to the screen if there is anyParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- query – string from flask.request.args[‘query’] which is applied to DATA using the query() function
Returns: JSON {success: True/False}
-
dtale.views.
update_column_position
(data_id)[source]¶ flask:flask.Flask
route to handle moving of columns within apandas.DataFrame
. Columns can be moved in one of these 4 directions: front, back, left, rightParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- action – string from flask.request.args[‘action’] of direction to move column
- col – string from flask.request.args[‘col’] of column name to move
Returns: JSON {success: True/False}
-
dtale.views.
update_formats
(data_id)[source]¶ flask:flask.Flask
route which updates the “columnFormats” property for global SETTINGS associated w/ the current portParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- all – boolean flag which, if true, tells us we should apply this formatting to all columns with the same data type as our selected column
- col – selected column
- format – JSON string for the formatting configuration we want applied to either the selected column of all columns with the selected column’s data type
Returns: JSON
-
dtale.views.
update_locked
(data_id)[source]¶ flask:flask.Flask
route to handle saving state associated with locking and unlocking columnsParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- action – string from flask.request.args[‘action’] of action to perform (lock or unlock)
- col – string from flask.request.args[‘col’] of column name to lock/unlock
Returns: JSON {success: True/False}
-
dtale.views.
update_settings
(data_id)[source]¶ flask:flask.Flask
route which updates global SETTINGS for current portParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- settings – JSON string from flask.request.args[‘settings’] which gets decoded and stored in SETTINGS variable
Returns: JSON
-
dtale.views.
update_visibility
(data_id)[source]¶ flask:flask.Flask
route to handle saving state associated visiblity of columns on the front-endParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- visibility (dict, optional) – string from flask.request.args[‘action’] of dictionary of visibility of all columns in a dataframe
- toggle (str, optional) – string from flask.request.args[‘col’] of column name whose visibility should be toggled
Returns: JSON {success: True/False}
-
dtale.views.
variance
(data_id)[source]¶ flask:flask.Flask
route which returns standard details about column data usingpandas.DataFrame.describe()
to the front-end as JSONParameters: - data_id (str) – integer string identifier for a D-Tale process’s data
- column – required dash separated string “START-END” stating a range of row indexes to be returned to the screen
Returns: JSON { describe: object representing output from
pandas.Series.describe()
, unique_data: array of unique values when data has <= 100 unique values success: True/False}
-
dtale.views.
view_code_popup
()[source]¶ flask:flask.Flask
route which serves up a base jinja template for code snippetsReturns: HTML
-
dtale.views.
view_iframe
(data_id=None)[source]¶ flask:flask.Flask
route which serves up base jinja template housing JS filesParameters: data_id (str) – integer string identifier for a D-Tale process’s data Returns: HTML
-
dtale.views.
view_main
(data_id=None)[source]¶ flask:flask.Flask
route which serves up base jinja template housing JS filesParameters: data_id (str) – integer string identifier for a D-Tale process’s data Returns: HTML
-
dtale.views.
view_main_by_name
(data_name=None)[source]¶ flask:flask.Flask
route which serves up base jinja template housing JS filesParameters: data_name (str) – integer string identifier for a D-Tale process’s data Returns: HTML
-
dtale.views.
view_network
(data_id=None)[source]¶ flask:flask.Flask
route which serves up base jinja template housing JS filesParameters: data_id (str) – integer string identifier for a D-Tale process’s data Returns: HTML