o
;ήcfb @ s< d dl Z d dlZd dlZd dlZd dlZd dlmZ d dlmZ d dl Z ddl
m Z m Z m
Z
mZmZ G dd dZG dd d Z
d%d e jd ed
edefddZd&defddZd&defddZd&defddZd&d e jdejfddZdejfddZd%d ej d edejfd d Z d d! Z d"d# Z e d$kre dS dS )' N)deque)IntEnum )get_producer_consumer_mapsis_fixed_size_tensor iterate_graph_per_graph_funciterate_graph_per_node_funcoptimize_modelc @ s( e Zd ZdZdd Zdd Zdd ZdS ) _SupportedOpsCheckerz
Class to process the md file with list of supported ops and caveats for an execution provider.
e.g. /tools/ci_build/github/android/nnapi_supported_ops.md
/tools/ci_build/github/apple/coreml_supported_ops.md
c C s || _ i | _t | _t|d9}| D ]+}|dr?| d}t |dkr?|d }|d }|
dd}d|v r?|| j|< qW d d S 1 sKw Y d S ) Nr| r z
:) _filename_opsset _ops_seenopen readlines
startswithstripsplitlenreplace)selffilenameflinepieces domain_opcaveat r# Y/tmp/pip-target-vg8gfxp4/lib/python/onnxruntime/tools/mobile_helpers/usability_checker.py__init__ s
"z _SupportedOpsChecker.__init__c C s< |j r|j nd}|d |j }|| jv }|r | j| |S )Nai.onnxr )domainop_typer r add)r noder' r! is_supportedr# r# r$ is_op_supported2 s
z$_SupportedOpsChecker.is_op_supportedc C s: g }t | jD ]}| j| }|r|| d| q|S )Nr )sortedr r append)r caveatsopr" r# r# r$ get_caveats< s
z _SupportedOpsChecker.get_caveatsN)__name__
__module__ __qualname____doc__r% r, r1 r# r# r# r$ r
s
r
c @ s@ e Zd ZG dd deZdd Zdd Zdejde fd d
Z
d S ) PartitioningInfoc @ s e Zd ZdZdZdZdS )zPartitioningInfo.TryWithEP)r )r r N)r2 r3 r4 NOMAYBEYESr# r# r# r$ TryWithEPG s r: c C s< d| _ d| _d| _d| _d | _g | _t | _d| _d| _ d S )N)
num_nodesnum_supported_nodesnum_partitionsnum_nodes_in_subgraphssupported_ops_checkersupported_groupsr unsupported_opsnodes_unsupported_due_to_op&nodes_unsupported_due_to_dynamic_input)r r# r# r$ r% L s
zPartitioningInfo.__init__c C sv | j | j }| j| d }| jdkr&|dkrtjjS |dkr"tjjS tjjS | jdkr7|dkr3tjjS tjjS tjjS )Nd r K 2 r ) r< r? r= r> r6 r: r9 r8 r7 )r r<
pct_supportedr# r# r$ suitabilityW s
z PartitioningInfo.suitabilityloggerep_namec
s | j | j }|| j d| j d| d| d | jr%|| j d | jrM|dddd | jD d
|d | j | jrM|d | j | t
j kr| j re|d
dt
| j | j }|rd |dd fdd |D | j| d }| jdkr|| d d'S | jdkr|dkr|| d|dd d'S |dkr|| d |dd d'S || d |d d d'S | jd!kr|dkr|| d"|dd# d'S || d$| j d%|dd& d'S )(z
Analyze the partitioning information and log the analysis
:param logger: Logger to use
:param ep_name: Execution provider name to use in the log messages
z partitions with a total of /z nodes can be handled by the z EP.z9 nodes are in subgraphs, which are currently not handled.zPartition sizes: [z, c S s g | ]}t t|qS r# )strr ).0 partitionr# r# r$
s z2PartitioningInfo.dump_analysis..]z"Unsupported nodes due to operator=z8Unsupported nodes due to input having a dynamic shape=%dzUnsupported ops: ,z zRCaveats that have not been checked and may result in a node not being supported: c s g | ] }t j | qS r# )oslinesep)rN r" indentr# r$ rP s rE r z$ cannot run any nodes in this model.r rF zD should work well for this model as there is one partition covering z.1fz % of the nodes in the model.rG z, may work well for this model, however only zD% of nodes will use it. Performance testing is required to validate.z4 will probably not work will for this model as only z.2fz% of nodes will use it.r zG can be considered for this model as there are two partitions covering z<% of the nodes. Performance testing is required to validate.z1 is not recommended with this model as there are z partitions covering zj% of the nodes in the model. This will most likely result in worse performance than just using the CPU EP.N)r< r? infor> r= rA joinrC rD getEffectiveLevelloggingDEBUGrB r- r@ r1 debug)r rJ rK r< r/ pct_nodes_using_epr# rV r$
dump_analysiss sn $
z PartitioningInfo.dump_analysisN) r2 r3 r4 r r: r% rI r[ LoggerrM r_ r# r# r# r$ r6 F s
r6 Fgraphr@ require_fixed_input_sizes
value_infoc s |rst dt| \}}dd | jD fdd dg i }t }t }| jD ]} | |v r8t|| nd}
|
|| < |
dkrG|| q,dtjd tjd
t gfd d } dg} t
| | | | d
g t d}
d}d}t }fdd}|s}|r|s| |}t }qx| } |
| }| pt fdd| jD }|o|}|s| v r|| qx|s|| jr| jnd d| j |d7 }n|d7 }|r|
d7 }
| | v r| | |v r|| D ]}| q| |v r|| D ]}|| }|d8 }|dkr
|| |||< q|s}|s}| t| |j
t| j}t}t }||_|
|_||_| d |_||_|_||_ ||_ ||_ |S )ah
Estimate the partitions the graph will be split into for nodes that is_node_supported_fn returns true for.
The check on whether a node is supported is purely based on the operator type. Additional limitations
(e.g. NNAPI EP only supports 2D Conv) are not checked, so partitions may not be 100% accurate. The limitations
for operators in the partitions are printed so the user can manually check.
:param graph: Graph to process
:param supported_ops_checker: Checker with info on supported ops.
:param require_fixed_input_sizes: If True, require that the inputs to a potentially supported node are
fixed size tensors for it to be considered as supported.
If True, onnx.shape_inference.infer_shapes should have been run on the model
to populate the shape information.
:param value_info: Map of value name to ValueInfoProto. Required if require_fixed_input_sizes is True to lookup
the shape of a value.
:return PartitioningInfo instance with details
zAvalue_info must be provided if require_fixed_input_sizes is True.c S s g | ]}|j qS r# )namerN ir# r# r$ rP s z&check_partitioning..c s$ | v r
t | S | v rdS dS )NTF)r )value) initializersrc r# r$ _is_fixed_shape_value s
z1check_partitioning.._is_fixed_shape_valueNr cur_graphoriginal_graphcountc S s& | |kr|d t | j7 < d S d S )Nr )r r* )rj rk rl r# r# r$ _count_subgraph_nodes s z1check_partitioning.._count_subgraph_nodes)rk rl c s< r p } | r d S d S N)r. copyclear)keep_partition)on_group_closed_fnsupported_groupsupported_group_borderrA r# r$ close_group s z'check_partitioning..close_groupc 3 s | ]} |V qd S rn r# re )ri r# r$ ( s z%check_partitioning..r& r r )
ValueErrorr initializerr r* r r. onnx
GraphProtointr r popleftr, allinputr) r' r( remover r6 r< r= r> r? r@ rA rB rC rD )ra r@ rb rc node_to_producersnode_to_consumers in_degreenodes_to_process nodes_to_process_with_next_groupr* node_input_edge_countrm nodes_in_subgraphsr= num_unsupported_nodes_due_to_op*num_unsupported_nodes_due_to_dynamic_inputrB ru r, is_input_shape_supportedis_node_supportedconsumerconsumer_node_in_degreer< r> rX r# )ri rh rr rs rt rA rc r$ check_partitioning s
"
3
r c C s t |}t| j||d u|}|S rn )r
r ra )modelsupported_ops_configrc
supported_opspartition_infor# r# r$ _check_ep_partitioningh s r c C P t tj}|d }| r|}n|jd }|d d d d d }t| ||S )Nznnapi_supported_ops.md toolsci_buildgithubandroidpathlibPath__file__parentexistsparentsr r rc
script_dir local_config config_pathort_rootr# r# r$ check_nnapi_partitionsn
r c C r )Nzcoreml_supported_ops.mdr r r r appler r r# r# r$ check_coreml_partitions| r r rJ c C sP d}d}g }| j D ]%}t|s*|| |r%|ddt| |d7 }q |d7 }q g }| jD ]%}t|sU|| |rP|ddt| |d7 }q4|d7 }q4| jsrt | j
dksrt |t | j ksr| d | jD ]}t|r|d7 }qu|d7 }qu|r|d| d| |r|r|d ||fS |d
||fS ) a
Check the shapes of graph inputs, values and graph outputs to determine if they have static or dynamic sizes.
NNAPI and CoreML do not support dynamically sized values.
:param graph: Graph to check. If shape inferencing has been run the checks on values will be meaningful.
:param logger: Optional logger for diagnostic information.
:return: Tuple of List of inputs with dynamic shapes, Number of dynamic values found
r z"Input is not a fixed size tensor: r r z#Output is not a fixed size tensor: ziUnable to check shapes within model. ONNX shape inferencing should be run on the model prior to checking.z Num values with fixed shape=z . Num values with dynamic shape=zModel has dynamic inputs and outputs. Consider re-exporting model with fixed sizes if NNAPI or CoreML can be used with this model.ao Model has dynamically sized inputs but fixed sized outputs.
If the sizes become fixed early in the model (e.g. pre-processing of a dynamic input size
results in a fixed input size for the majority of the model) performance with NNAPI and CoreML,
if applicable, should not be significantly impacted.) r~ r r. rX rY rM r outputrc r r* warning) ra rJ num_fixed_valuesnum_dynamic_valuesdynamic_inputsrf dynamic_outputsovir# r# r$ check_shapes sP
&
r c s t | }t j|i jjD ]}||j< qjjD ]}||j< q jjD ]}||j< q)t j\ } fdd}|dt
}|dt }|t j
jksW|t j
jkrc tjkrcd d |t j
jkps|t j
jkS )Nc s d| |} tjkr|| | } d| d|j |tjj krp rp d |} tjkrM d || | } d| d|j |tjj
krh d |j |j krp|}|S )N Checking z Model should perform well with z as is: zHChecking if model will perform better if the dynamic shapes are fixed...zHPartition information if the model was updated to make the shapes fixed:z) if modified to have fixed input shapes: zPShapes can be altered using python -m onnxruntime.tools.make_dynamic_shape_fixed) rX rZ r[ r\ r_ rI rd r6 r: r9 r7 rg )rK checker_funcr rI partition_info_with_fixed_shapesfixed_shape_suitabilityr rJ model_with_shape_infovalue_to_shaper# r$ check_ep s.
zchecker..check_epNNAPICoreMLzKRe-run with log level of DEBUG for more details on the NNAPI/CoreML issues.z---------------)ry loadshape_inference infer_shapesra r~ rd r rc r r r r6 r: r9 rZ r[ r\ rX r7 )
model_pathrJ r vr r nnapi_suitabilitycoreml_suitabilityr# r r$ checker s,
r r
skip_optimizec C s |s
t d}|t j |d| d t %}|s,t|| j }t
| | |} t t | j
dd|}W d |S 1 sBw Y |S )a
Analyze the provided model to determine if it's likely to work well with the NNAPI or CoreML Execution Providers
:param model_path: Model to analyze.
:param skip_optimize: Skip optimizing to BASIC level before checking. When exporting to ORT format we will do this
optimization..
:param logger: Logger for output
:return: True if either the NNAPI or CoreML Execution Providers may work well with this model.
usability_checkerr z for usability with ORT Mobile.T)strictN)r[ getLoggersetLevelINFOrX tempfileTemporaryDirectoryr r rd r r rM resolve)r r rJ tmptmp_pathtry_epsr# r# r$
analyze_model s
r c C sV t jtjtdd} | jdg dddd | jdd d
d | jd tjd
d | S )Nz3Analyze an ONNX model for usage with the ORT mobile) descriptionz --log_level)r] rX r errorrX z
Logging level)choicesdefaulthelpz--skip_optimize
store_truezDon't optimize the model to BASIC level prior to analyzing. Optimization will occur when exporting the model to ORT format, so in general should not be skipped unless you have a specific reason to do so.)actionr r zProvide path to ONNX model)typer )
argparseArgumentParserrT pathbasenamer add_argumentr r
parse_args)parserr# r# r$ r ) s r c C s t } td}| jdkr|tj n | jdkr |tj n| jdkr,|tj n|tj | j
}t || j | d S )Nr r] rX r )
r r[ r log_levelr r\ r WARNINGERRORr r r r )argsrJ r r# r# r$ run_analyze_model= s
r __main__)FNrn )!r r[ rT r r collectionsr enumr ry onnx_model_utilsr r r r r r
r6 rz booldictr r r r r` r r r r r r r2 r# r# r# r$ sD /y
-I <