class VBench
Bases: BaseMetric
Source code in aigve/metrics/multi_aspect_metrics/vbench/vbench_metric.py
__init__(collect_device=None, prefix=None, vbench_prompt_json_path=None, eval_aspects=None, eval_mode='vbench_standard', local=False, read_frame=False, category='', imaging_quality_preprocessing_mode='longer', **kwargs)
Parameters:
Name | Type | Description | Default |
---|---|---|---|
collect_device
|
Optional[Union[str, device]]
|
The device to collect the data on. |
None
|
prefix
|
Optional[str]
|
The prefix to use for the metric. |
None
|
vbench_prompt_json_path
|
str
|
The path to the vbench prompt JSON file. |
None
|
eval_aspects
|
list
|
the evaluation aspects, if the vbench_prompt_json_path is not None, the available aspects are |
None
|
eval_mode
|
str
|
the evaluation mode, if the vbench_prompt_json_path is not None, the available modes are ['vbench_standard', 'vbench_category'] if the vbench_prompt_json_path is None, the available modes are ['custom_input'] |
'vbench_standard'
|
local
|
bool
|
whether to use local mode, if True, the model will be loaded locally, if False, the model will be loaded from the internet |
False
|
read_frame
|
bool
|
whether to read the frame from the video, if True, the model will read the frame from the video, if False, the model will not read the frame from the video |
False
|
category(str)
|
The category to evaluate on, usage: --category=animal. |
required | |
imaging_quality_preprocessing_mode(str)
|
|
required | |
2.
|
longer
|
if the longer side is more than 512, the image is resized so that the longer side is 512. |
required |
3.
|
shorter_centercrop
|
if the shorter side is more than 512, the image is resized so that the shorter side is 512. |
required |
4.
|
None
|
no preprocessing |
required |
Source code in aigve/metrics/multi_aspect_metrics/vbench/vbench_metric.py
compute_metrics(results)
Parameters:
Name | Type | Description | Default |
---|---|---|---|
results
|
list
|
The results to compute the metrics from. |
required |
process(data_batch, data_samples)
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data_batch
|
Any
|
The data batch to process. |
required |
data_samples
|
Sequence[dict]
|
The data samples to process. |
required |