o 8ήcʔ@s6dZddlZddlmZddlZddlmZddlmZm Z m Z m Z ddl Z edZedZedZeejZeZedZedZedZedZd d Zdd d Zd dZiZdddZifddZGddde Z!Gddde!Z"Gddde!Z#Gddde Z$dddZ%d d!Z&dd"d#Z'd$d%Z(d&d'Z)d(d)Z*dd*d+Z+d,d-Z,d.d/Z-d0d1Z.d2d3Z/d4d5Z0d6d7Z1d8d9Z2d:d;Z3e4dd?Z6eddAdBZ7edCdDZ8eddFdGZ9edHdIZ:ddJdKZ;dLdMZddRdSZ? ddTdUZ@dVdWZAdXdYZBdZd[ZCd\d]ZDeCZEeBZFd^d_ZGd`daZHddbdcZIeddddeZJeHZKdfdgZLdhdiZMdjdkZNdldmZOdndoZPddpdqZQdrdsZRdtduZSddwdxZTdydzZUd{d|ZVd}d~ZWddZXdddZYdddZZddZ[ddZ\ddZ]ddZ^ddZ_ddZ`ddZaddZbdS)z+ Generic helpers for LLVM code generation. N)contextmanager)ir)utilstypesconfig debuginfo cC|d||dS)N!=r) icmp_unsignedtypebuildervaluer9/tmp/pip-target-vg8gfxp4/lib/python/numba/core/cgutils.py as_bool_bitrcCsJ|durtdd|D}|tj}t|D] \}}||||}q|S)zH Create an anonymous struct containing the given LLVM *values*. NcSg|]}|jqSrr.0vrrr %z)make_anonymous_struct..rLiteralStructType Undefined enumerate insert_value)rvalues struct_type struct_valirrrrmake_anonymous_struct s  r&cCs*t|}t|}tttd||S)z0 Make a byte array constant from *buf*. r ) bytearraylenrConstant ArrayTypeIntType)bufbnrrrmake_bytearray,sr/rcCsb||f}t|}|dur/ttd|}|jdt|}|f}t|d}t|||}|t|<|S)zK Returns a specialized StructProxy subclass for the given fe_type. N)rdata_)_fe_type)_struct_proxy_cachegetValueStructProxyDataStructProxy__name__strdictr)fe_typekind cache_keyresbaseclsnamebases clsmembersrrrcreate_struct_proxy8s   rBcCsV|}|jjD]}||t||}t|||q|D] \}}t|||q|S)zJ Copy structure from *src* to *dst* with replacement from *repl*. )copy _datamodel_fieldspopgetattrsetattritems)dstsrcreplkrrrr copy_structKs rNcseZdZdZdZd!ddZddZddZd d Zd d Z d dZ ddZ ddZ fddZ ddZddZddZddZddZdd ZZS)" _StructProxyz Creates a `Structure` like interface that is constructed with information from DataModel instance. FE type must have a data model that is a subclass of StructModel. NcCs||_|jj|j|_t|jtjjjst d |j||_ | |j|_ t|j r-J||\}}|jj|j krGtd|j |jf|durd|j|jjkr]td|jj|jf|j ||||_||_dS)NzNot a structure model: {0}z!bad ref type: expected %s, got %sz#bad value type: expected %s, got %s)_contextdata_model_managerr2rD isinstancenumbacore datamodel StructModel TypeErrorformat_builder _get_be_type_be_type is_pointer _make_refsrpointeeAssertionError as_pointerstore_value _outer_ref)selfcontextrrref outer_refrrr__init__cs,   z_StructProxy.__init__cCs"|dur t|j|jdd}||fS)z Return an (outer ref, value ref) pair. By default, these are the same pointers, but a derived class may override this. NTzfill) alloca_oncerYr[)rdrfrrrr]|sz_StructProxy._make_refscCtNNotImplementedErrorrdrUrrrrZz_StructProxy._get_be_typecCrlrmrnrdindexvalrrr_cast_member_to_valuerqz"_StructProxy._cast_member_to_valuecCrlrmrnrrrrr_cast_member_from_valuerqz$_StructProxy._cast_member_from_valuecCst|j|jd|SNr) gep_inboundsrYrbrdrsrrr_get_ptr_by_indexsz_StructProxy._get_ptr_by_indexcCs|j|}||Srm)rDget_field_positionrz)rdattrnamersrrr_get_ptr_by_names  z_StructProxy._get_ptr_by_namecCs"|ds ||j|St|z; Load the LLVM value of the named *field*. r1) startswithrDr{AttributeErrorrdfieldrrr __getattr__s z_StructProxy.__getattr__cs0|drtt|||S|||j|<dSz@ Store the LLVM *value* into the named *field*. r1N)rsuperrO __setattr__rDr{rdrr __class__rrrs z_StructProxy.__setattr__cCs|j||}|||Sz> Load the LLVM value of the field at *index*. )rYloadrzru)rdrs member_valrrr __getitem__s z_StructProxy.__getitem__cCs||}|||}|j|jjkr>t|jr3t|jjr3|jj|jjjkr3|j|j||jjj}n t dj ||||d|j ||dS)C Store the LLVM *value* into the field at *index*. zjInvalid store of {value.type} to {ptr.type.pointee} in {self._datamodel} (trying to write member #{index}))rptrrdrsN) rzrvrr^r\rP addrspacecastrY addrspacerWrXra)rdrsrrrrr __setitem__s   z_StructProxy.__setitem__cCs|jjSz. Return the number of fields. )rD field_countrdrrr__len__sz_StructProxy.__len__cC|jSzF Return the LLVM pointer to the underlying structure. )rcrrrr _getpointerz_StructProxy._getpointercC|j|jSzM Load and return the value of the underlying LLVM structure. )rYrrcrrrr _getvaluez_StructProxy._getvaluecC>t|jrJ|j|jksJ|j|jf|j||jdS)z4 Store the value in this structure. N)r\rr[rYrarbrdrrrr _setvaluesz_StructProxy._setvalueNN)r7 __module__ __qualname____doc__r2rhr]rZrurvrzr}rrrrrrrr __classcell__rrrrrOZs$   rOc@(eZdZdZddZddZddZdS) r5zl Create a StructProxy suitable for accessing regular values (e.g. LLVM values or alloca slots). cC|Srm)get_value_typerprrrrZzValueStructProxy._get_be_typecC|Srmrrrrrrrurqz&ValueStructProxy._cast_member_to_valuecCrrmrrrrrrrvrqz(ValueStructProxy._cast_member_from_valueNr7rrrrZrurvrrrrr5s  r5c@r) r6zO Create a StructProxy suitable for accessing data persisted in memory. cCrrm) get_data_typerprrrrZrzDataStructProxy._get_be_typecC|j|}||j|Srm)rD get_model from_datarYrdrsrtmodelrrrru z%DataStructProxy._cast_member_to_valuecCrrm)rDras_datarYrrrrrvrz'DataStructProxy._cast_member_from_valueNrrrrrr6s  r6csreZdZdZdddZddZdd Zd d Zfd d ZddZ ddZ ddZ ddZ ddZ ddZZS) Structurezs A high-level object wrapping a alloca'ed LLVM structure, including named fields and attribute access. NFc Cs0|||_||_||_|dur:t||jdd|_|dur9t|jr$J|j|jks2J|j|jf|||jn.|dus@Jt|jsGJ|j|jj kre|rZ| ||j }n t d|jj |jf||_i|_ g|_g|_td}t|jD]\}\}} ||j |<|j|t|f|j| qzdS)NTriz-mismatching pointer type: got %s, expected %sr)get_struct_type_typerPrYrkrbr\rrar^bitcastr`rW_namemap_fdmap_typemapint32_tr rEappend) rdrerrrfcast_refr>r%rMtprrrrhs<    zStructure.__init__cCs|jj|j|j|dd}|S)NT)inbounds)rYgeprbr)rdrsrrrrrz$szStructure._get_ptr_by_indexcCs||j|Srm)rzr)rdr|rrrr}(zStructure._get_ptr_by_namecCs |ds ||j|St|r~)rrrrrrrr+s zStructure.__getattr__cs.|drtt|||S|||j|<dSr)rrrrrrrrrr4s zStructure.__setattr__cCs|j||Sr)rYrrzryrrrr<szStructure.__getitem__cCsN||}|jj|jkrd}t||t|jjt|jf|j||dS)rz:Type mismatch: __setitem__(%d, ...) expected %r but got %rN)rzrr^r_r8rYra)rdrsrrfmtrrrrCs  zStructure.__setitem__cCs t|jSr)r(rrrrrrOs zStructure.__len__cCrr)rbrrrrrUrzStructure._getpointercCrr)rYrrbrrrrr[rzStructure._getvaluecCr)z!Store the value in this structureN)r\rrrYrarbrrrrraszStructure._setvalue)NNF)r7rrrrhrzr}rrrrrrrrrrrrrrs   rFc Cst|tr tt|}t|:||j|||d}| |d|Wdn1s0wY|rA| |j d||WdS1sMwYdS)aAllocate stack memory at the entry block of the current function pointed by ``builder`` with llvm type ``ty``. The optional ``size`` arg set the number of element to allocate. The default is 1. The optional ``name`` arg set the symbol name inside the llvm IR for debugging. If ``zfill`` is set, fill the memory with zeros at the current use-site location. Note that the memory is always zero-filled after the ``alloca`` at init-site (the entry block). )sizenameN) rRintrr)intp_trsuspend_emissiongoto_entry_blockallocararr^)rtyrrrjrrrrrkjs   $rkcCs$|d}|tdg}||tS)zCompute sizeof using GEP Nr)rrptrtointr)rptr_typenulloffsetrrrsizeofs rcCs t||j|d}||||S)z Like alloca_once(), but passing a *value* instead of a type. The type is inferred and the allocated slot is also initialized with the given value. ri)rkrra)rrrrjstoragerrralloca_once_values rcCs(t|||}|jd|jd|S)z_ Insert a pure function (in the functional programming sense) in the given module. readonlynounwind)get_or_insert_function attributesaddmodulefntyrfnrrrinsert_pure_functions   rcCs(|j|d}|durt|||}|S)zm Get the function named *name* with type *fnty* from *module*, or insert it if it doesn't exist. N)globalsr4rFunctionrrrrrsrcCs*z||WSty||YSwrm)get_named_metadataKeyErroradd_named_metadata)rrrrrget_or_insert_named_metadatas   rcCs||}t||||Srm)get_unique_namerGlobalVariable)rrrr unique_namerrradd_global_variables rcCs"|j}|jdur||dSdSrm) basic_block terminatorbranch)rbbendbbrrr terminates rcCs|dSrmr)ltyperrrget_null_valuerrcCt|j}|d||S)N==rrr rrtrrrris_null rcCr)Nr rrrrr is_not_nullrrcC|j|ddS)NFlikelyif_thenrpredrrr if_unlikelyrcCr)NTrrrrrr if_likelyrrcCs|||Srm)rnot_rrrrifnotrrcCs|d}|j||dgdS)z# Increment an index *val*. rnsw)flags)rr)rrtonerrrincrement_indexs rLoop)rsdo_breakc #s*|dur|j}|dur|d}|}d}d}dfdd}j}||j|dd } d | |} | |Wdn1sUwY|t| |Vj}t | } t |Wdn1s}wY| ||| | | dS) a Generate LLVM IR for a for-loop in [start, count). *start* is equal to 0 by default. Yields a Loop namedtuple with the following members: - `index` is the loop index's value - `do_break` is a no-argument callable to break out of the loop Nrfor.condfor.bodyfor.endcsdSrm)rrrrrrrrzfor_range..do_break loop.indexr<) rappend_basic_blockrr goto_blockphi icmp_signedcbranchrrr add_incomingposition_at_end) rcountstartintpstopbbcondbbbodyrbbstartrsrincrrr r for_ranges4            rTccsT|dur|j}|d}|d}|d}|j} ||||.|j|dd} |j|dd} |r=|d| |} n|d | |} || ||Wdn1sUwY|| | | fV|j}|| |} t || }t ||Wdn1swY| || | | || t |d | | ||||dS) a[ Generate LLVM IR for a for-loop based on a slice. Yields a (index, count) tuple where `index` is the slice index's value inside the loop, and `count` the iteration count. Parameters ------------- builder : object IRBuilder object start : int The beginning value of the slice stop : int The end value of the slice step : int The step value of the slice intp : The data type inc : boolean, optional Signals whether the step is positive (True) or negative (False). Returns ----------- None Nr r r r rz loop.countr>r)rrrrrrrrrrrrrr)r)rrrsteprincrrrrrsrrr next_countrrrfor_range_slices8            r$c cs|j}|d|t|d}t|||||dd}t|||||dd}tdd}|j|dd\} } || ||| |fVWd d S1sIwYd S) a A helper wrapper for for_range_slice(). This is a context manager which yields two for_range_slice()-alike context managers, the first for the positive step case, the second for the negative step case. Use: with for_range_slice_generic(...) as (pos_range, neg_range): with pos_range as (idx, count): ... with neg_range as (idx, count): ... >=rT)r"Fc ssf|&| }|VWdn1swYWddSWddS1s,wYdSrmr)condinner_cmrrrrcm_cond`s"z(for_range_slice_generic..cm_condrN)rrrr)r$rif_else) rrrr!r is_pos_step pos_for_range neg_for_ranger(then otherwiserrrfor_range_slice_genericLs "r/Cccs|dvsJ|sdVdS|dkrdd}ndd}t||||}t|t|ks.J||VWddS1s>wYdS)a Generate a loop nest walking a N-dimensional array. Yields a tuple of N indices for use in the inner loop body, iterating over the *shape* space. If *order* is 'C' (the default), indices are incremented inside-out (i.e. (0,0), (0,1), (0,2), (1,0) etc.). If *order* is 'F', they are incremented outside-in (i.e. (0,0), (1,0), (2,0), (0,1) etc.). This has performance implications when walking an array as it impacts the spatial locality of memory accesses. CFrFcSs|dddS)Nrxrrr~szloop_nest..cSs|Srmrr4rrrr6sN) _loop_nestr()rshaperorder_swapindicesrrr loop_nestjs    "r<c cst||d|d>}t|dkr3t||dd|}|jf|VWdn1s-wYn |jfVWddSWddS1sKwYdS)Nrrr)rr(r7rs)rr8rloopr;rrrr7s  "r7cCsPt|}|dur |dj}t||tj}t|D] \}}||||}q|S)z Pack a sequence of values in a LLVM array. *ty* should be given if the array may be empty, in which case the type can't be inferred from the values. Nr)r(rrr*rr r!)rr"rr.aryr%rrrr pack_arrays r@cCsBtdd|D}|tj}t|D] \}}||||}q|S)z7 Pack a sequence of values into a LLVM struct. cSrrrrrrrrrzpack_struct..r)rr"structtystr%rrrr pack_structs  rCcs0|dur tjj}fddt|D}|S)zH Unpack an array or structure of values, return a Python tuple. Ncsg|]}|qSr) extract_value)rr%rtuprrrsz unpack_tuple..)r(relementsrange)rrFrvalsrrEr unpack_tuples   rJc CsBt||j|jd}t||j|jd}t|||j|||j|||d S)N)r)r0r8strideslayoutinds wraparound boundscheck)rJr8ndimrKget_item_pointer2r0rL) rerarytyr?rMrNrOshapesrKrrrget_item_pointers  rTc sfdd}d}d}t|tjr||jt|fWdn1s1wYdd}t|tjrL||jt|fWddS1s`wYdS)NcsPdurttrtddStddStddS)NzFdebug: IndexError: index %d is out of bounds for axis {} with size %d zFdebug: IndexError: index %d is out of bounds for axis %d with size %d z9debug: IndexError: index %d is out of bounds for size %d )rRrprintfrXraxisrdimlenindrr_dbgs  zdo_boundscheck.._dbgzindex is out of boundsr%rr)rrrFULL_TRACEBACKS call_convreturn_user_exc IndexErrorr) rerrYrXrWrZmsgout_of_bounds_upperout_of_bounds_lowerrrVrdo_boundschecks  "rbc s|r+g} t||D] \} } d| | d} | | } | | | }| |q n|} |rEtt| |D]\}\} } t|| | |q6| sP|t dgS| dj}|dvrg}|dkrt t |D]}|d}||ddD]} ||}qs||qen*|dkrt t |D]}|d}|d|D]} ||}q||qnt d|d}t| |D]\}} ||}||}q||g}|Sfdd t|| D}tj|}t||S) Nrrr1r0rr2 unreachablecsg|] \}}||qSr)mul)rsr%rrrrsz%get_item_pointer2..)ziprrrselectrr rbrrrHr(rd Exception functoolsreduce pointer_add)rerr0r8rKrLrMrNrOr;rYrXnegativewrappedselectedrWrstepsr%lastjlocretmprdimoffsrrrfrrQsR       rQcCs\|d}t|jtjtjfr|||}|St|jtjr&||||}|Std|jf)Nrzunexpected value type %s)rrRr FloatType DoubleTyper+rrW)rrfpredicondnullvalisnullrrr_scalar_pred_against_zeros  r|cCt||t|jddS)zK Return a predicate representing whether *value* is equal to zero. rr|rjpartial fcmp_orderedrrrris_scalar_zero%rcCr})z Return a predicate representing whether a *value* is not equal to zero. (not exactly "not is_scalar_zero" because of nans) r r|rjrfcmp_unorderedrrrris_not_scalar_zero-rcCr})z] Return a predicate representing whether *value* is equal to either zero or NaN. rrrrrris_scalar_zero_or_nan6rrcCr})z: Is *value* negative? Assumes *value* is signed. rr~rrrr is_scalar_negCrrcCsb|jt||dd|d}|ddpd}|j|||WddS1s*wYdS)zu Guard against *value* being null or zero. *exc_tuple* should be a (exception type, arguments...) tuple. FrrrN)rrr\r])rerr exc_tupleexcexc_argsrrr guard_nullKs "rcCspt|jtjs J|j|r|fnd}|jt||dd|j|t|WddS1s1wYdS)zG Guard against *pointer* being NULL (and raise a MemoryError). rFrN) rRrr PointerTyperrr\r] MemoryError)rerpointerr_rrrrguard_memory_errorVs "rccsB|jt|||d dVWddS1swYdS)z> Execute the given block if the scalar value is zero. rN)rr)rrrrrrif_zero`s"rcCs t|tjS)z7 Whether the LLVM type *typ* is a struct type. )rRrr)ltyprrrr\ls r\cCs.t||d|}t|jjrJ|||Srw)rxr\rr^rr`)rrecordrtyppvalrrrget_record_memberssrcCr )Nrr)rr)rrtrrr is_neg_intyrrcOst||g|Rddi|S)z8 Same as *gep*, but add the `inbounds` keyword. rT)r)rrrMkwsrrrrx}srxc Osb|dd}|dd}|rJg}|D]}t|tr t|}n|}||q|j||||dS)z Emit a getelementptr instruction for the given pointer and indices. The indices can be LLVM values or Python int constants. rrrF)rr)rFrRrrrr) rrrMrrridxr%rYrrrrs     rcCs<||t}t|trt|}|||}|||p|jS)z Add an integral *offset* to pointer *ptr*, and return a pointer of *return_type* (or, if omitted, the same type as *ptr*). Note the computation is done in bytes, and ignores the width of the pointed item type. )rrrRrrinttoptrr)rrr return_typeintptrrrrrls   rlcCsN|jdt|jf}||t}t|trt|}|||||t dgdS)z= Fill *size* bytes starting from *ptr* with *value*. z llvm.memsetrN) rdeclare_intrinsic voidptr_trrrRrint8_tcallbool_t)rrrrrrrrmemsets   rcCs0||}t||t||jd|||dS)z7 Fill padding bytes of the pointee with zeros. rN)rrrrra)rrrtrrrmemset_paddings rinternalcCs<t|tjr |}n|j}t||j|}||_d|_||_|S)zO Get or create a (LLVM module-)global constant with *name* or *value*. T) rRrModulerrrlinkageglobal_constant initializer)builder_or_modulerrrrr0rrrrs rc Cs|dksJ||}|d}t||j}|t||P\}}||||}|||Wdn1s:wY||||}|||}|||||Wdn1sbwYWdn1sqwY||}||| ||} || fS)a  Compute the (quotient, remainder) of *val* divided by the constant positive *divisor*. The semantics reflects those of Python integer floor division, rather than C's / LLVM's signed division and modulo. The difference lies with a negative *val*. rrN) rrkr)rsdivrarsubrrd) rrtdivisorrquotif_negif_posquot_val val_plus_onerem_valrrrdivmod_by_constants&        rcCs&|d}|||||||S)z Branch conditionally or continue. Note: a new block is created and builder is moved to the end of the new block. z .continue)rrr)rr&bbtruebbcontrrrcbranch_or_continues  rcCsx|j|jksJt|||jd"}|||jg}|||jg}||||WddS1s5wYdS)z Emit a memcpy to the builder. Copies each element of dst to src. Unlike the C equivalent, each element can be any LLVM type. Assumes ------- * dst.type == src.type * count is positive r=N)rrrrsrar)rrJrKrr>out_ptrin_ptrrrrmemcpys "rc Csd|j}t|trt||}|j|tt|g}t} | || |t| |t| ||| gdSrm) rrRrrr)rrr false_bitrrrd) r func_namerJrKritemsizealignsize_tr is_volatilerrr _raw_memcpy s     rcCt|d|||||S)za Emit a raw memcpy() call for `count` items of size `itemsize` from `src` to `dest`. z llvm.memcpyrrrJrKrrrrrr raw_memcpysrcCr)zb Emit a raw memmove() call for `count` items of size `itemsize` from `src` to `dest`. z llvm.memmoverrrrr raw_memmove#s rc CsX|||}||d}||d}|||}||d}||||d} || fS)zq Compute (a * b + c) and return a (result, overflow bit) pair. The operands must be signed integers. rr)smul_with_overflowrDsadd_with_overflowor_) rar-cpprodprod_ovfrer=ovfrrrmuladd_with_overflow,s     rc Gst|tsJ|j}t}t|dd}t|d|}tjt |gdd}z| d}Wnt y<tj ||dd}Ynw| ||} ||| gt|S)a Calls printf(). Argument `format` is expected to be a Python string. Values to be printed are listed in `args`. Note: There is no checking to ensure there is correct number of values in `args` and there type matches the declaration in the format string. ascii printf_formatTvar_argrUr)rRr8rrr/encoderr FunctionTyper get_globalrrrrlist) rrXargsmodcstring fmt_bytes global_fmtrrptr_fmtrrrrU:s    rUc Gst|tsJ|j}t}t|dd}t|d|}tjt |t |gdd} d} t j r/d| } z| | } WntyGtj|| | d} Ynw|||} || ||| gt|S) z8Calls libc snprintf(buffer, bufsz, format, ...args) rrsnprintf_formatTrsnprintfr1r)rRr8rrr/rrrrrrrIS_WIN32rrrrrr) rbufferbufszrXrrrrrrsymbolrrrrrrTs$    rcGsXt|tsJttd|}t||dd}||t}t||t ||g|R|S)zwSimilar to `snprintf()` but the buffer is stack allocated to size *bufsz*. Returns the buffer pointer as i8*. r Tri) rRrrr*r+rkrrrr)rrrXrspacetyspacerrrrsnprintf_stackbufferns  rcCs|ddS)zk Normalize the given string to latin1 compatible encoding that is suitable for use in LLVM IR. utf8latin1)rdecode)textrrrnormalize_ir_text{src Csd}||t}t|d||td}|||}t||G}||j t|}| d|td}| | t|dWdn1sIwY| ||j g}| |} t|d| Wdn1skwYt|ddS) zIDebug print the memory region in *ptr* to *ptr + nbytes* as hex. zhexdump p=%p n=%zur rr Nz %02x)zextrrUrr+rr`ruremrsr rrr) rrnbytesbytes_per_linebyte_trdiv_by do_new_linerrtrrrhexdumps$       rcCs |dupt|tjpt|tjS)z returns if 'ty' is none N)rRrNoneTypeOmitted)rrrr is_nonelikes   rcCstt|t||S)z Create an LLVM-constant of a fixed-length array from Python values. The type provided is the type of the elements. )rr)r*r()rrtrrrcreate_constant_arraysrrm)r)NrF)rF)rr)NT)r0)FF)F)r)r)cr collections contextlibrrjllvmliter numba.corerrrrnumba.core.datamodelrSr+rrr MACHINE_BITSrr`rtrue_bitr true_byte false_byterr&r/r3rBrNobjectrOr5r6rrkrrrrrrrrrrrrrr namedtuplerrr$r/r<r7r@rCrJrTrbrQr|rrris_trueis_falserrrr guard_zeror\rrrxrrlrrrrrrrrrrrUrrrrrrrrrrs           l      + 9        :