flip.nvflare.controllers.fed_evaluation
Classes
FLIP model-collection evaluation: evaluate a set of server-provided models on every client. |
Module Contents
- class flip.nvflare.controllers.fed_evaluation.ModelEval(task_check_period=0.5, submit_model_timeout=600, validation_timeout: int = 6000, model_locator_id='', formatter_id='', submit_model_task_name=AppConstants.TASK_SUBMIT_MODEL, evaluation_task_name=PTConstants.EvalTaskName, cleanup_models=False, participating_clients=None, wait_for_clients_timeout=300)
Bases:
nvflare.app_common.workflows.cross_site_model_eval.CrossSiteModelEvalFLIP model-collection evaluation: evaluate a set of server-provided models on every client.
Subclasses NVFLARE’s
CrossSiteModelEvalto inherit its constructor validation, unknown-task routing, and stop logic. Unlike cross-site validation, where each client’s submitted model is validated by every other client, evaluation loads a collection of models server-side and bundles them into a single evaluation task broadcast to all clients.Sharing the base also defines the
_validation_task_name/_cross_val_dirattributes the previous standalone fork referenced but never assigned.- _evaluation_task_name = 'evaluation'
- _eval_results_dir = 'evaluation_results'
- _eval_results: dict
- _all_models_dxo = None
- start_controller(fl_ctx: nvflare.apis.fl_context.FLContext)
- control_flow(abort_signal: nvflare.apis.signal.Signal, fl_ctx: nvflare.apis.fl_context.FLContext)
- _locate_server_models(fl_ctx: nvflare.apis.fl_context.FLContext) bool
Load the whole collection of models to evaluate from the model locator.
Unlike stock (one DXO located per model name), evaluation loads every model in a single
locate_modelcall and keeps the collection DXO to broadcast to clients.
- _before_send_validate_task_cb(client_task: nvflare.apis.controller_spec.ClientTask, fl_ctx: nvflare.apis.fl_context.FLContext)
Bundle the whole model collection into the task data sent to each client.
- _accept_val_result(client_name: str, result: nvflare.apis.shareable.Shareable, fl_ctx: nvflare.apis.fl_context.FLContext)
Record each client’s evaluation result, keyed flatly by client.
- handle_event(event_type: str, fl_ctx: nvflare.apis.fl_context.FLContext)