:py:mod:`api.metrics`
=====================

.. py:module:: api.metrics


Module Contents
---------------

Classes
~~~~~~~

.. autoapisummary::

   api.metrics.ProjectMetrics



Functions
~~~~~~~~~

.. autoapisummary::

   api.metrics.calculate_metrics



Attributes
~~~~~~~~~~

.. autoapisummary::

   api.metrics._dt_fmt
   api.metrics.logger


.. py:data:: _dt_fmt
   :value: '%Y-%m-%d %H:%M:%S.%f'

   

.. py:data:: logger

   

.. py:function:: calculate_metrics(project_ids, report_name)

   Computes metrics in a background task
   :param projects: list of projects to compute metrics for.
       Uses the 'first' for the CDB / vocab or ModelPack,
       but should be the same CDB, but will still try and compute metrics regardless.
   :return: computed metrics results


.. py:class:: ProjectMetrics(mct_export_data, cat)


   Bases: :py:obj:`object`

   Class to analyse MedCATtrainer exports

   .. py:method:: __init__(mct_export_data, cat)

      :param mct_export_paths: List of paths to MedCATtrainer exports


   .. py:method:: _annotations()


   .. py:method:: annotation_df()

      DataFrame of all annotations created
      :return: DataFrame


   .. py:method:: concept_summary(extra_cui_filter=None)

      Summary of only correctly annotated concepts from a mct export
      :return: DataFrame summary of annotations.


   .. py:method:: enrich_medcat_metrics(examples)

      Add the user prop to the medcat output metrics. Can potentially add more later for each of the categories


   .. py:method:: user_stats(by_user = True)

      Summary of user annotation work done

      :param by_user: User Stats grouped by user rather than day
      :return: DataFrame of user annotation work done


   .. py:method:: rename_meta_anns(meta_anns2rename=dict(), meta_ann_values2rename=dict())

      TODO: the meta_ann_values2rename has issues
      :param meta_anns2rename: Example input: `{'Subject/Experiencer': 'Subject'}`
      :param meta_ann_values2rename: Example input: `{'Subject':{'Relative':'Other'}}`
      :return:


   .. py:method:: _eval_model(model, data, config)


   .. py:method:: _eval(metacat_model, mct_export)


   .. py:method:: full_annotation_df()

      DataFrame of all annotations created including meta_annotation predictions.
      This function is similar to annotation_df with the addition of Meta_annotation predictions from the medcat model.
      prerequisite Args: MedcatTrainer_export([mct_export_paths], model_pack_path=<path to medcat model>)


   .. py:method:: meta_anns_concept_summary()

      Calculate performance metrics for meta annotations per concept.

      Returns:
          List[Dict]: List of dictionaries containing concept-level meta annotation metrics


   .. py:method:: generate_report(meta_ann=False)