o i @sNddlZddlmZddlmZddlmZmZmZddl m Z m Z e e ZeGdddZeGdd d Zd eeeeefd eed eefd dZGdddeZGdddeZGdddeZdeeeefd eeeeffddZddZd ed eed eeeeffddZdeeed eeefddZdS)N) defaultdict) dataclass)AnyOptionalUnion)logging yaml_dumpc@s8eZdZUdZeed<eed<eed<eed<eed<dZeeed<dZ eeed <dZ eeed <dZ eeed <dZ ee eefed <dZeeed <dZeeed<dZee eefed<dZeeed<dZeeed<dZeeed<dZeeed<edefddZdddefddZdddZdS) EvalResultu Flattened representation of individual evaluation results found in model-index of Model Cards. For more information on the model-index spec, see https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1. Args: task_type (`str`): The task identifier. Example: "image-classification". dataset_type (`str`): The dataset identifier. Example: "common_voice". Use dataset id from https://hf.co/datasets. dataset_name (`str`): A pretty name for the dataset. Example: "Common Voice (French)". metric_type (`str`): The metric identifier. Example: "wer". Use metric id from https://hf.co/metrics. metric_value (`Any`): The metric value. Example: 0.9 or "20.0 ± 1.2". task_name (`str`, *optional*): A pretty name for the task. Example: "Speech Recognition". dataset_config (`str`, *optional*): The name of the dataset configuration used in `load_dataset()`. Example: fr in `load_dataset("common_voice", "fr")`. See the `datasets` docs for more info: https://hf.co/docs/datasets/package_reference/loading_methods#datasets.load_dataset.name dataset_split (`str`, *optional*): The split used in `load_dataset()`. Example: "test". dataset_revision (`str`, *optional*): The revision (AKA Git Sha) of the dataset used in `load_dataset()`. Example: 5503434ddd753f426f4b38109466949a1217c2bb dataset_args (`dict[str, Any]`, *optional*): The arguments passed during `Metric.compute()`. Example for `bleu`: `{"max_order": 4}` metric_name (`str`, *optional*): A pretty name for the metric. Example: "Test WER". metric_config (`str`, *optional*): The name of the metric configuration used in `load_metric()`. Example: bleurt-large-512 in `load_metric("bleurt", "bleurt-large-512")`. See the `datasets` docs for more info: https://huggingface.co/docs/datasets/v2.1.0/en/loading#load-configurations metric_args (`dict[str, Any]`, *optional*): The arguments passed during `Metric.compute()`. Example for `bleu`: max_order: 4 verified (`bool`, *optional*): Indicates whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not. Automatically computed by Hugging Face, do not set. verify_token (`str`, *optional*): A JSON Web Token that is used to verify whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not. source_name (`str`, *optional*): The name of the source of the evaluation result. Example: "Open LLM Leaderboard". source_url (`str`, *optional*): The URL of the source of the evaluation result. Example: "https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard". task_type dataset_type dataset_name metric_type metric_valueN task_namedataset_config dataset_splitdataset_revision dataset_args metric_name metric_config metric_argsverified verify_token source_name source_urlreturncCs|j|j|j|j|jfS)z9Returns a tuple that uniquely identifies this evaluation.)r r rrrselfrc/var/www/html/karishye-ai-python/venv/lib/python3.10/site-packages/huggingface_hub/repocard_data.pyunique_identifiers zEvalResult.unique_identifierothercCsD|jD]\}}|dkrq|dkrt||t||krdSqdS)zx Return True if `self` and `other` describe exactly the same metric but with a different value. rrFT)__dict__itemsgetattr)rr!key_rrris_equal_except_valuesz EvalResult.is_equal_except_valuecCs$|jdur|jdurtddSdS)NzAIf `source_name` is provided, `source_url` must also be provided.)rr ValueErrorrrrr __post_init__szEvalResult.__post_init__)rN)__name__ __module__ __qualname____doc__str__annotations__rrrrrrrdictrrrrboolrrrpropertytupler r'r)rrrrr s. 3 r c@seZdZdZd"defddZddZdd Zd#d ee e d e fd dZ ddZ ddZ d$de ded efddZd$de ded efddZde d efddZde ded d fddZde d efddZd efd d!Zd S)%CardDataaStructure containing metadata from a RepoCard. [`CardData`] is the parent class of [`ModelCardData`] and [`DatasetCardData`]. Metadata can be exported as a dictionary or YAML. Export can be customized to alter the representation of the data (example: flatten evaluation results). `CardData` behaves as a dictionary (can get, pop, set values) but do not inherit from `dict` to allow this export step. Fignore_metadata_errorscKs|j|dSN)r"update)rr5kwargsrrr__init__szCardData.__init__cCs(t|j}||dd|DS)zConverts CardData to a dict. Returns: `dict`: CardData represented as a dictionary ready to be dumped to a YAML block for inclusion in a README.md file. cSsi|] \}}|dur||qSr6r).0r%valuerrr sz$CardData.to_dict..)copydeepcopyr"_to_dictr#r data_dictrrrto_dicts  zCardData.to_dictcCsdS)zUse this method in child classes to alter the dict representation of the data. Alter the dict in-place. Args: data_dict (`dict`): The raw dict representation of the card data. Nrr@rrrr?szCardData._to_dictNoriginal_orderrcsH|rfdd|ttjt|D_td|dS)a Dumps CardData to a YAML block for inclusion in a README.md file. Args: line_break (str, *optional*): The line break to use when dumping to yaml. Returns: `str`: CardData represented as a YAML block. cs"i|] }|jvr|j|qSrr")r:krrrr<s   z$CardData.to_yaml..F) sort_keys line_break)listsetr"keysrrBstrip)rrGrCrrrto_yamls  zCardData.to_yamlcC t|jSr6)reprr"rrrr__repr__s zCardData.__repr__cCs|Sr6)rLrrrr__str__szCardData.__str__r%defaultcCs|j|}|dur |S|S)#Get value for a given metadata key.N)r"get)rr%rQr;rrrrSs z CardData.getcCs|j||S)z#Pop value for a given metadata key.)r"pop)rr%rQrrrrTz CardData.popcCs |j|S)rRrDrr%rrr __getitem__ zCardData.__getitem__r;cCs||j|<dS)z#Set value for a given metadata key.NrD)rr%r;rrr __setitem__rUzCardData.__setitem__cCs ||jvS)z%Check if a given metadata key is set.rDrVrrr __contains__rXzCardData.__contains__cCrM)z'Return the number of metadata keys set.)lenr"rrrr__len__rXzCardData.__len__)F)NNr6)r*r+r,r-r1r9rBr?rrHr.rLrOrPrrSrTrWrYrZintr\rrrrr4s  r4 eval_results model_namercCs`|durgSt|tr|g}t|trtdd|Ds&tdt|d|dur.td|S)Ncss|]}t|tVqdSr6) isinstancer )r:rrrr sz)_validate_eval_results..zM`eval_results` should be of type `EvalResult` or a list of `EvalResult`, got .z7Passing `eval_results` requires `model_name` to be set.)r`r rHallr(type)r^r_rrr_validate_eval_resultss rfcseZdZdZdddddddddddddd deeeeefdeeeeefdeeedeeeeefd eed eed eed eed eeedeedeedeeede ffddZ ddZ Z S) ModelCardDataaQModel Card Metadata that is used by Hugging Face Hub when included at the top of your README.md Args: base_model (`str` or `list[str]`, *optional*): The identifier of the base model from which the model derives. This is applicable for example if your model is a fine-tune or adapter of an existing model. The value must be the ID of a model on the Hub (or a list of IDs if your model derives from multiple models). Defaults to None. datasets (`Union[str, list[str]]`, *optional*): Dataset or list of datasets that were used to train this model. Should be a dataset ID found on https://hf.co/datasets. Defaults to None. eval_results (`Union[list[EvalResult], EvalResult]`, *optional*): List of `huggingface_hub.EvalResult` that define evaluation results of the model. If provided, `model_name` is used to as a name on PapersWithCode's leaderboards. Defaults to `None`. language (`Union[str, list[str]]`, *optional*): Language of model's training data or metadata. It must be an ISO 639-1, 639-2 or 639-3 code (two/three letters), or a special value like "code", "multilingual". Defaults to `None`. library_name (`str`, *optional*): Name of library used by this model. Example: keras or any library from https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/src/model-libraries.ts. Defaults to None. license (`str`, *optional*): License of this model. Example: apache-2.0 or any license from https://huggingface.co/docs/hub/repositories-licenses. Defaults to None. license_name (`str`, *optional*): Name of the license of this model. Defaults to None. To be used in conjunction with `license_link`. Common licenses (Apache-2.0, MIT, CC-BY-SA-4.0) do not need a name. In that case, use `license` instead. license_link (`str`, *optional*): Link to the license of this model. Defaults to None. To be used in conjunction with `license_name`. Common licenses (Apache-2.0, MIT, CC-BY-SA-4.0) do not need a link. In that case, use `license` instead. metrics (`list[str]`, *optional*): List of metrics used to evaluate this model. Should be a metric name that can be found at https://hf.co/metrics. Example: 'accuracy'. Defaults to None. model_name (`str`, *optional*): A name for this model. It is used along with `eval_results` to construct the `model-index` within the card's metadata. The name you supply here is what will be used on PapersWithCode's leaderboards. If None is provided then the repo name is used as a default. Defaults to None. pipeline_tag (`str`, *optional*): The pipeline tag associated with the model. Example: "text-classification". tags (`list[str]`, *optional*): List of tags to add to your model that can be used when filtering on the Hugging Face Hub. Defaults to None. ignore_metadata_errors (`str`): If True, errors while parsing the metadata section will be ignored. Some information might be lost during the process. Use it at your own risk. kwargs (`dict`, *optional*): Additional metadata that will be added to the model card. Defaults to None. Example: ```python >>> from huggingface_hub import ModelCardData >>> card_data = ModelCardData( ... language="en", ... license="mit", ... library_name="timm", ... tags=['image-classification', 'resnet'], ... ) >>> card_data.to_dict() {'language': 'en', 'license': 'mit', 'library_name': 'timm', 'tags': ['image-classification', 'resnet']} ``` NF) base_modeldatasetsr^language library_namelicense license_name license_linkmetricsr_ pipeline_tagtagsr5rhrir^rjrkrlrmrnror_rprqr5c  sL||_||_||_||_||_||_||_||_| |_| |_ | |_ t | |_ | dd}|rezt|\} }| |_ ||_Wn(ttfyd}z| rNtdn td|jd|dWYd}~nd}~wwtjdi||jrz t|j|j |_WdSty}z| rtd|dntd||WYd}~dSd}~wwdS) N model-indexz1T', and 'other'. source_datasets (`list[str]]`, *optional*): Indicates whether the dataset is an original dataset or extended from another existing dataset. Options are: 'original' and 'extended'. task_categories (`Union[str, list[str]]`, *optional*): What categories of task does the dataset support? task_ids (`Union[str, list[str]]`, *optional*): What specific tasks does the dataset support? paperswithcode_id (`str`, *optional*): ID of the dataset on PapersWithCode. pretty_name (`str`, *optional*): A more human-readable name for the dataset. (ex. "Cats vs. Dogs") train_eval_index (`dict`, *optional*): A dictionary that describes the necessary spec for doing evaluation on the Hub. If not provided, it will be gathered from the 'train-eval-index' key of the kwargs. config_names (`Union[str, list[str]]`, *optional*): A list of the available dataset configs for the dataset. NF)rjrlannotations_creatorslanguage_creatorsmultilingualitysize_categoriessource_datasetstask_categoriestask_idspaperswithcode_id pretty_nametrain_eval_index config_namesr5rjrlrrrrrrrrrrrr5c sp||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ | p+| dd|_ tjdi|dS)Ntrain-eval-indexr)rrrjrlrrrrrrrrrTrr{r9)rrjrlrrrrrrrrrrrr5r8rrrr9szDatasetCardData.__init__cCs|d|d<dS)Nrr)rTr@rrrr?szDatasetCardData._to_dict) r*r+r,r-rrr.rHr0r1r9r?rrrrrrs\*     $rcseZdZdZddddddddddddd deedeedeedeed eed eed eed eed eeedeeedeeedeffddZ Z S) SpaceCardDataa Space Card Metadata that is used by Hugging Face Hub when included at the top of your README.md To get an exhaustive reference of Spaces configuration, please visit https://huggingface.co/docs/hub/spaces-config-reference#spaces-configuration-reference. Args: title (`str`, *optional*) Title of the Space. sdk (`str`, *optional*) SDK of the Space (one of `gradio`, `streamlit`, `docker`, or `static`). sdk_version (`str`, *optional*) Version of the used SDK (if Gradio/Streamlit sdk). python_version (`str`, *optional*) Python version used in the Space (if Gradio/Streamlit sdk). app_file (`str`, *optional*) Path to your main application file (which contains either gradio or streamlit Python code, or static html code). Path is relative to the root of the repository. app_port (`str`, *optional*) Port on which your application is running. Used only if sdk is `docker`. license (`str`, *optional*) License of this model. Example: apache-2.0 or any license from https://huggingface.co/docs/hub/repositories-licenses. duplicated_from (`str`, *optional*) ID of the original Space if this is a duplicated Space. models (list[`str`], *optional*) List of models related to this Space. Should be a dataset ID found on https://hf.co/models. datasets (`list[str]`, *optional*) List of datasets related to this Space. Should be a dataset ID found on https://hf.co/datasets. tags (`list[str]`, *optional*) List of tags to add to your Space that can be used when filtering on the Hub. ignore_metadata_errors (`str`): If True, errors while parsing the metadata section will be ignored. Some information might be lost during the process. Use it at your own risk. kwargs (`dict`, *optional*): Additional metadata that will be added to the space card. Example: ```python >>> from huggingface_hub import SpaceCardData >>> card_data = SpaceCardData( ... title="Dreambooth Training", ... license="mit", ... sdk="gradio", ... duplicated_from="multimodalart/dreambooth-training" ... ) >>> card_data.to_dict() {'title': 'Dreambooth Training', 'sdk': 'gradio', 'license': 'mit', 'duplicated_from': 'multimodalart/dreambooth-training'} ``` NF) titlesdk sdk_versionpython_versionapp_fileapp_portrlduplicated_frommodelsrirqr5rrrrrrrlrrrirqr5c  s\||_||_||_||_||_||_||_||_| |_| |_ t | |_ t j di| dS)Nr)rrrrrrrlrrrirtrqr{r9)rrrrrrrrlrrrirqr5r8rrrr9 s zSpaceCardData.__init__) r*r+r,r-rr.r]rHr1r9rrrrrrsN4     rr}cCsg}|D]}|d}|d}|D]}|dd}|dd}|dd}|dd} |dd} |dd} |dd} |dd } |d id}|d id }|d D]`}|d}|d }|d}|d }|d}|d}|d}td!id|d|d| d|d|d|d| d| d| d| d|d|d|d|d|d|d|}||q]qq||fS)"aTakes in a model index and returns the model name and a list of `huggingface_hub.EvalResult` objects. A detailed spec of the model index can be found here: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1 Args: model_index (`list[dict[str, Any]]`): A model index data structure, likely coming from a README.md file on the Hugging Face Hub. Returns: model_name (`str`): The name of the model as found in the model index. This is used as the identifier for the model on leaderboards like PapersWithCode. eval_results (`list[EvalResult]`): A list of `huggingface_hub.EvalResult` objects containing the metrics reported in the provided model_index. Example: ```python >>> from huggingface_hub.repocard_data import model_index_to_eval_results >>> # Define a minimal model index >>> model_index = [ ... { ... "name": "my-cool-model", ... "results": [ ... { ... "task": { ... "type": "image-classification" ... }, ... "dataset": { ... "type": "beans", ... "name": "Beans" ... }, ... "metrics": [ ... { ... "type": "accuracy", ... "value": 0.9 ... } ... ] ... } ... ] ... } ... ] >>> model_name, eval_results = model_index_to_eval_results(model_index) >>> model_name 'my-cool-model' >>> eval_results[0].task_type 'image-classification' >>> eval_results[0].metric_type 'accuracy' ``` nameresultstaskredatasetconfigsplitrevisionargssourceurlror;r verifyTokenr r r r rrrrrrrrrrrrNr)rSr append)r}r^elemrrresultr rr r rrrrrrmetricr rrrrrr eval_resultrrrru+s|8               )rucCsNt|tttfrt|dd|DSt|tr%t|dd|DS|S)zk Recursively remove `None` values from a dict. Borrowed from: https://stackoverflow.com/a/20558778 css |] }|durt|VqdSr6 _remove_none)r:xrrrrbsz_remove_none..css4|]\}}|dur|durt|t|fVqdSr6r)r:rEvrrrrbs2)r`rHr3rIrer0r#)objrrrrs  rc Cstt}|D] }||j|qg}|D]@}|d}|j|jd|j|j|j |j |j |j ddd|Dd}|j durRd|j i}|jdurN|j|d <||d <||q||d g} t| S) aTakes in given model name and list of `huggingface_hub.EvalResult` and returns a valid model-index that will be compatible with the format expected by the Hugging Face Hub. Args: model_name (`str`): Name of the model (ex. "my-cool-model"). This is used as the identifier for the model on leaderboards like PapersWithCode. eval_results (`list[EvalResult]`): List of `huggingface_hub.EvalResult` objects containing the metrics to be reported in the model-index. Returns: model_index (`list[dict[str, Any]]`): The eval_results converted to a model-index. Example: ```python >>> from huggingface_hub.repocard_data import eval_results_to_model_index, EvalResult >>> # Define minimal eval_results >>> eval_results = [ ... EvalResult( ... task_type="image-classification", # Required ... dataset_type="beans", # Required ... dataset_name="Beans", # Required ... metric_type="accuracy", # Required ... metric_value=0.9, # Required ... ) ... ] >>> eval_results_to_model_index("my-cool-model", eval_results) [{'name': 'my-cool-model', 'results': [{'task': {'type': 'image-classification'}, 'dataset': {'name': 'Beans', 'type': 'beans'}, 'metrics': [{'type': 'accuracy', 'value': 0.9}]}]}] ``` r)rer)rrerrrrc Ss.g|]}|j|j|j|j|j|j|jdqS))rer;rrrrr)r rrrrrr)r:rrrr s z/eval_results_to_model_index..)rrroNrrr)rr)rrHr rvaluesr rr r rrrrrrr) r_r^task_and_ds_types_maprmodel_index_datar sample_resultdatarr}rrrrs@%      rrqcCs0|dur|Sg}|D] }||vr||q |Sr6)r)rq unique_tagstagrrrrts rt)r= collectionsr dataclassesrtypingrrrhuggingface_hub.utilsrr get_loggerr*rxr r4rHr.rfrgrrr0r3rurrrtrrrrs4   S P*Qh& &\