o ;ήc @s$ddlZeeZGdddZdS)Nc@s<eZdZdZed defddZeddZedd Zd S) PastKeyValuesHelperzEHelper functions to process past key values for encoder-decoder modelFpresentcCsg}g}t|D]4}||rd|d|gn d|d|g||r1d|d|gn d|d|gq||S) Npresent_key_self_present_value_self_past_key_self_past_value_self_present_key_cross_present_value_cross_past_key_cross_past_value_cross_)rangeextend) num_layersrpast_self_namespast_cross_namesirU/tmp/pip-target-vg8gfxp4/lib/python/onnxruntime/transformers/models/t5/past_helper.pyget_past_namess z"PastKeyValuesHelper.get_past_namesc Cshg}g}t|D]'\}}t|dksJdt||\}}}}|||g|||gq||fS)aSplit present state from grouped by layer to grouped by self/cross attention. Before: (past_key_self_0, past_value_self_0, past_key_cross_0, past_value_cross_0), (past_key_self_1, past_value_self_1, past_key_cross_1, past_value_cross_1), ... After: (past_key_self_0, past_value_self_0, past_key_self_1, past_value_self_1, ...), (past_key_cross_0, past_value_cross_0, past_key_cross_1, past_value_cross_1, ...) z!Expected to have four items. Got ) enumeratelenr ) present_key_values present_self present_crossrpresent_layer_ipresent_key_selfpresent_value_selfpresent_key_crosspresent_value_crossrrrgroup_by_self_or_cross sz*PastKeyValuesHelper.group_by_self_or_crosscs0tdks JtfddtDS)aReorder past state from grouped by self/cross attention to grouped by layer. Before: past_key_self_0, past_value_self_0, past_key_self_1, past_value_self_1, ..., past_key_cross_0, past_value_cross_0, past_key_cross_1, past_value_cross_1, ... After: (past_key_self_0, past_value_self_0, past_key_cross_0, past_value_cross_0), (past_key_self_1, past_value_self_1, past_key_cross_1, past_value_cross_1), rc3sT|]%}d|d|ddd|dd|dgVqdS)Nr).0rrpastrr <s  z5PastKeyValuesHelper.group_by_layer..)rtupler )r%rrr$rgroup_by_layer5sz"PastKeyValuesHelper.group_by_layerN)F) __name__ __module__ __qualname____doc__ staticmethodboolrr r(rrrrr s r)logging getLoggerr)loggerrrrrrs