o ;ήc9 @sddlZddlZddlZddlZddlmZddlZddlZddlm Z ddl m Z ddl m Z ddlmZddlmZddlmZmZejejejed d dd lmZdd lmZdaed d d d d d d d d ddZededej dddZ!ddZ"    d)ddZ#     d*ddZ$      d+ddZ%dede&d e'fd!d"Z(d,de&d#e&fd$d%Z)d&d'Z*e+d(kre!Z,e*e,dSdS)-N)Path)PRETRAINED_LONGFORMER_MODELS) load_model)version)register_custom_op_symbolic) parse_args)LongformerModelLongformerSelfAttentionz..) BertOnnxModel)torch_onnx_exportvic Cs|jd||||||||| d S)Nz"com.microsoft::LongformerAttention) num_heads_iwindow_i)op) ginputweightbiasmask global_weight global_bias global_mask num_headswindowra/tmp/pip-target-vg8gfxp4/lib/python/onnxruntime/transformers/models/longformer/convert_to_onnx.pymy_longformer_attention9s rz onnxruntime::LongformerAttention za./torch_extensions/build/lib.linux-x86_64-3.8/longformer_attention.cpython-38-x86_64-linux-gnu.soc Cst}|jdddtdddtd|jddd d d |jdd |jd dd dd |jdd|jdddd dd |jdd|jdddtdddgdd|}|S)z:Parse arguments Returns: args: Namespace z-mz--modelFzlongformer-base-4096z=Checkpoint directory or pre-trained model names in the list: z, )requiredtypedefaulthelpz--export_padding store_truez{Export padding logic to ONNX graph. If not enabled, user need pad input so that sequence length is multiple of window size.)ractionr")export_paddingz--no_merge_qkvzFStack the weights of q, k and v on dimension 0 instead of dimension 1.) no_merge_qkvz-oz--optimize_onnxz(Use optimizer.py to optimize onnx model.) optimize_onnxz-pz --precisionfp32fp16zLPrecision of model to run: fp32 for full precision, fp16 for mixed precision)rr r!choicesr") argparseArgumentParser add_argumentstrjoinrkeys set_defaultsr)parserargsrrrparse_arguments]sX      r4cCs|r |jddn|jd}t|d|}tj|jtj|d}d|dd|df<tj|jtj|d}d|dddf<|||fS)Nr)dtypedevice) attention_windowtorcharange unsqueezetoonesshapelongzeros)configr%r7sequence_length input_idsattention_maskglobal_attention_maskrrrget_dummy_inputss rFc CsP|}t|jdkr$|jddkr$|dkr$|dddks&J|}||d}t|jdkrP|jddkrP|dkrP|dddksRJtj |j j dd|j j dd|jj ddftd}tdkr|||jd|j}tj |jj dd|jj dd|jj ddftd} tdkr| |jd|j} tdkrtj |j j|j j|jjfdd} | d|j} tj |jj|jj|jjfdd} | d|j} n*tj |j j|j j|jj|jj|jjfdd} | d|j} |jj} | d|j} tjj||| || | ||j|j } | |ks#Jd | f} | S) Nrr5g)dimzUnexpected size)intlenr> count_nonzeroitemtolistfloat masked_fillr9stackqueryr transposekeyvalueweight_bias_formatreshape embed_dim query_global key_global value_globalrops onnxruntimeLongformerAttentionrone_sided_attn_window_sizesize)self hidden_statesrDis_index_maskedis_index_global_attnis_global_attnr input_maskrrrr attn_outputoutputsrrr&my_longformer_self_attention_forward_4s|         " rkFcCs|dksJt||||||SNFrk)rcrdrDrerfrgoutput_attentionsrrr(my_longformer_self_attention_forward_4_3s rocCs*|dksJ|dus Jt||||||Srlrm)rcrdrDlayer_head_maskrerfrgrnrrr*my_longformer_self_attention_forward_4_3_2s rqmodelonnx_model_pathr%c Cs t|j|tdd\}}}||||d}ttjtdkr$tdd t t j j}tttd}||vrFtdt t j jtd t j } ||t _ |||f} t|jjd d d t|| |d gd ddgddddddddddddddddddidtd|| t _ dS)aExport longformer model to ONNX Args: model (LongformerModel): longformer model onnx_model_path (str): output onnx path export_padding (bool): whether export padding logic to ONNX so that input string can be any length. Raises: RuntimeError: This tool requires transformers 4.0.0 or later. RuntimeError: LongformerSelfAttention.forward arguments are different. cpu)r7)rDrEz4.0.0z/This tool requires transformers 4.0.0 or later. )zwself hidden_states attention_mask layer_head_mask is_index_masked is_index_global_attn is_global_attn output_attentionszgself hidden_states attention_mask is_index_masked is_index_global_attn is_global_attn output_attentionszUself hidden_states attention_mask is_index_masked is_index_global_attn is_global_attnzCurrent argumentsztLongformerSelfAttention.forward arguments are different. Please install supported version (like transformers 4.3.0).T)parentsexist_ok )rCrDrE last_statepooler batch_sizerB)rr5)rCrDrEryrzz com.microsoftr5) opset_version input_names output_names dynamic_axes custom_opsetszONNX model exported to N)rFrAr9r7rparse transformers __version__ RuntimeErrorr/inspectgetfullargspecr forwardr3rqrorkprintrparentmkdirr ) rrrsr%rCrDrE_rV args_to_funcoriginal_forwardexample_inputsrrrexport_longformer.sX      rfp32_model_pathcCsrt|ddd}t|}|d}|r |||td||r7|jdd|||td|dSdS)aOptimize longformer onnx model Args: onnx_model_path (str): path of original ONNX model. fp32_model_path (str): path of optimized fp32 model. fp16_model_path (str, optional): path of optimized fp16 model. Defaults to None. NT)formatload_external_dataFzoptimized fp32 model saved to ) keep_io_typeszoptimized fp16 model saved to )rr optimizesave_model_to_filerconvert_float_to_float16)rsrfp16_model_pathrr optimizeruse_external_data_formatrrroptimize_longformerws   rcCs|j}|d}|jr dndatt|}t|||j|js$|j dkrE|dtd}|j dkr;|dtdnd}t |||dSdS) Nz.onnxrr5r(_fz _fp32.onnxr)z _fp16.onnx) rrr&rXrfrom_pretrainedrrr%r' precisionr)r3 model_namersrrrrrrrmains r__main__)NNNN)NNNNF)NNNNNF)N)-r+rossyspathlibrr9rlongformer_helperronnxr packagingr torch.onnxrtorch.onnx.symbolic_helperrrr pathappendr/dirname__file__onnx_model_bertr torch_onnx_export_helperr rXrr^ load_libraryr4rFrkrorqr.boolrrr__name__r3rrrrsd!           : \  I