METADATA 69 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099110011011102110311041105110611071108110911101111111211131114111511161117111811191120112111221123112411251126112711281129113011311132113311341135113611371138113911401141114211431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177117811791180118111821183118411851186118711881189119011911192119311941195119611971198119912001201120212031204120512061207120812091210121112121213121412151216121712181219122012211222122312241225122612271228122912301231123212331234123512361237123812391240124112421243124412451246124712481249125012511252125312541255125612571258125912601261126212631264126512661267126812691270127112721273127412751276127712781279128012811282128312841285128612871288128912901291129212931294129512961297129812991300130113021303130413051306130713081309131013111312131313141315131613171318131913201321132213231324132513261327132813291330133113321333133413351336133713381339134013411342134313441345134613471348134913501351135213531354135513561357135813591360136113621363136413651366136713681369137013711372137313741375137613771378137913801381138213831384138513861387138813891390139113921393139413951396139713981399140014011402140314041405140614071408140914101411141214131414141514161417141814191420142114221423142414251426142714281429143014311432143314341435143614371438143914401441144214431444144514461447144814491450145114521453145414551456145714581459146014611462146314641465146614671468146914701471147214731474147514761477147814791480148114821483148414851486148714881489149014911492149314941495149614971498149915001501150215031504150515061507150815091510151115121513151415151516151715181519152015211522152315241525152615271528152915301531153215331534153515361537
  1. Metadata-Version: 2.4
  2. Name: simsimd
  3. Version: 6.5.16
  4. Summary: Portable mixed-precision BLAS-like vector math library for x86 and ARM
  5. Home-page: https://github.com/ashvardanian/simsimd
  6. Author: Ash Vardanian
  7. Author-email: 1983160+ashvardanian@users.noreply.github.com
  8. License: Apache-2.0
  9. Classifier: License :: OSI Approved :: Apache Software License
  10. Classifier: Operating System :: POSIX :: Linux
  11. Classifier: Operating System :: Microsoft :: Windows
  12. Classifier: Operating System :: MacOS
  13. Classifier: Development Status :: 5 - Production/Stable
  14. Classifier: Natural Language :: English
  15. Classifier: Intended Audience :: Developers
  16. Classifier: Intended Audience :: Information Technology
  17. Classifier: Programming Language :: C
  18. Classifier: Programming Language :: Python :: Implementation :: CPython
  19. Classifier: Programming Language :: Python :: 3
  20. Classifier: Programming Language :: Python :: 3.9
  21. Classifier: Programming Language :: Python :: 3.10
  22. Classifier: Programming Language :: Python :: 3.11
  23. Classifier: Programming Language :: Python :: 3.12
  24. Classifier: Programming Language :: Python :: 3.13
  25. Classifier: Programming Language :: Python :: 3.14
  26. Classifier: Programming Language :: Python :: Free Threading :: 3 - Stable
  27. Classifier: Topic :: Scientific/Engineering :: Mathematics
  28. Classifier: Topic :: Scientific/Engineering :: Information Analysis
  29. Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
  30. Classifier: Topic :: Scientific/Engineering :: Chemistry
  31. Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
  32. Description-Content-Type: text/markdown
  33. License-File: LICENSE
  34. Dynamic: author
  35. Dynamic: author-email
  36. Dynamic: classifier
  37. Dynamic: description
  38. Dynamic: description-content-type
  39. Dynamic: home-page
  40. Dynamic: license
  41. Dynamic: license-file
  42. Dynamic: summary
  43. ![SimSIMD banner](https://github.com/ashvardanian/ashvardanian/blob/master/repositories/SimSIMD.jpg?raw=true)
  44. Computing dot-products, similarity measures, and distances between low- and high-dimensional vectors is ubiquitous in Machine Learning, Scientific Computing, Geospatial Analysis, and Information Retrieval.
  45. These algorithms generally have linear complexity in time, constant or linear complexity in space, and are data-parallel.
  46. In other words, it is easily parallelizable and vectorizable and often available in packages like BLAS (level 1) and LAPACK, as well as higher-level `numpy` and `scipy` Python libraries.
  47. Ironically, even with decades of evolution in compilers and numerical computing, [most libraries can be 3-200x slower than hardware potential][benchmarks] even on the most popular hardware, like 64-bit x86 and Arm CPUs.
  48. Moreover, most lack mixed-precision support, which is crucial for modern AI!
  49. The rare few that support minimal mixed precision, run only on one platform, and are vendor-locked, by companies like Intel and Nvidia.
  50. SimSIMD provides an alternative.
  51. 1️⃣ SimSIMD functions are practically as fast as `memcpy`.
  52. 2️⃣ Unlike BLAS, most kernels are designed for mixed-precision and bit-level operations.
  53. 3️⃣ SimSIMD often [ships more binaries than NumPy][compatibility] and has more backends than most BLAS implementations, and more high-level interfaces than most libraries.
  54. [benchmarks]: https://ashvardanian.com/posts/simsimd-faster-scipy
  55. [compatibility]: https://pypi.org/project/simsimd/#files
  56. <div>
  57. <a href="https://pepy.tech/project/simsimd">
  58. <img alt="PyPI" src="https://static.pepy.tech/personalized-badge/simsimd?period=total&units=abbreviation&left_color=black&right_color=blue&left_text=SimSIMD%20Python%20installs" />
  59. </a>
  60. <a href="https://www.npmjs.com/package/simsimd">
  61. <img alt="npm" src="https://img.shields.io/npm/dy/simsimd?label=JavaScript%20NPM%20installs" />
  62. </a>
  63. <a href="https://crates.io/crates/simsimd">
  64. <img alt="rust" src="https://img.shields.io/crates/d/simsimd?label=Rust%20Crate%20installs" />
  65. </a>
  66. <img alt="GitHub code size in bytes" src="https://img.shields.io/github/languages/code-size/ashvardanian/simsimd">
  67. <a href="https://github.com/ashvardanian/SimSIMD/actions/workflows/release.yml">
  68. <img alt="GitHub Actions Ubuntu" src="https://img.shields.io/github/actions/workflow/status/ashvardanian/SimSIMD/release.yml?branch=main&label=Ubuntu&logo=github&color=blue">
  69. </a>
  70. <a href="https://github.com/ashvardanian/SimSIMD/actions/workflows/release.yml">
  71. <img alt="GitHub Actions Windows" src="https://img.shields.io/github/actions/workflow/status/ashvardanian/SimSIMD/release.yml?branch=main&label=Windows&logo=windows&color=blue">
  72. </a>
  73. <a href="https://github.com/ashvardanian/SimSIMD/actions/workflows/release.yml">
  74. <img alt="GitHub Actions macOS" src="https://img.shields.io/github/actions/workflow/status/ashvardanian/SimSIMD/release.yml?branch=main&label=macOS&logo=apple&color=blue">
  75. </a>
  76. <a href="https://github.com/ashvardanian/SimSIMD/actions/workflows/release.yml">
  77. <img alt="GitHub Actions CentOS Linux" src="https://img.shields.io/github/actions/workflow/status/ashvardanian/SimSIMD/release.yml?branch=main&label=CentOS&logo=centos&color=blue">
  78. </a>
  79. </div>
  80. ## Features
  81. __SimSIMD__ (Arabic: "سيمسيم دي") is a mixed-precision math library of __over 350 SIMD-optimized kernels__ extensively used in AI, Search, and DBMS workloads.
  82. Named after the iconic ["Open Sesame"](https://en.wikipedia.org/wiki/Open_sesame) command that opened doors to treasure in _Ali Baba and the Forty Thieves_, SimSIMD can help you 10x the cost-efficiency of your computational pipelines.
  83. Implemented distance functions include:
  84. - Euclidean (L2) and Cosine (Angular) spatial distances for Vector Search. _[docs][docs-spatial]_
  85. - Dot-Products for real & complex vectors for DSP & Quantum computing. _[docs][docs-dot]_
  86. - Hamming (~ Manhattan) and Jaccard (~ Tanimoto) bit-level distances. _[docs][docs-binary]_
  87. - Set Intersections for Sparse Vectors and Text Analysis. _[docs][docs-sparse]_
  88. - Mahalanobis distance and Quadratic forms for Scientific Computing. _[docs][docs-curved]_
  89. - Kullback-Leibler and Jensen–Shannon divergences for probability distributions. _[docs][docs-probability]_
  90. - Fused-Multiply-Add (FMA) and Weighted Sums to replace BLAS level 1 functions. _[docs][docs-fma]_
  91. - For Levenshtein, Needleman–Wunsch, and Smith-Waterman, check [StringZilla][stringzilla].
  92. - 🔜 Haversine and Vincenty's formulae for Geospatial Analysis.
  93. [docs-spatial]: #cosine-similarity-reciprocal-square-root-and-newton-raphson-iteration
  94. [docs-curved]: #curved-spaces-mahalanobis-distance-and-bilinear-quadratic-forms
  95. [docs-sparse]: #set-intersection-galloping-and-binary-search
  96. [docs-binary]: https://github.com/ashvardanian/SimSIMD/pull/138
  97. [docs-dot]: #complex-dot-products-conjugate-dot-products-and-complex-numbers
  98. [docs-probability]: #logarithms-in-kullback-leibler--jensenshannon-divergences
  99. [docs-fma]: #mixed-precision-in-fused-multiply-add-and-weighted-sums
  100. [scipy]: https://docs.scipy.org/doc/scipy/reference/spatial.distance.html#module-scipy.spatial.distance
  101. [numpy]: https://numpy.org/doc/stable/reference/generated/numpy.inner.html
  102. [stringzilla]: https://github.com/ashvardanian/stringzilla
  103. Moreover, SimSIMD...
  104. - handles `float64`, `float32`, `float16`, and `bfloat16` real & complex vectors.
  105. - handles `int8` integral, `int4` sub-byte, and `b8` binary vectors.
  106. - handles sparse `uint32` and `uint16` sets, and weighted sparse vectors.
  107. - is a zero-dependency [header-only C 99](#using-simsimd-in-c) library.
  108. - has [Python](#using-simsimd-in-python), [Rust](#using-simsimd-in-rust), [JS](#using-simsimd-in-javascript), and [Swift](#using-simsimd-in-swift) bindings.
  109. - has Arm backends for NEON, Scalable Vector Extensions (SVE), and SVE2.
  110. - has x86 backends for Haswell, Skylake, Ice Lake, Genoa, and Sapphire Rapids.
  111. - with both compile-time and runtime CPU feature detection easily integrates anywhere!
  112. Due to the high-level of fragmentation of SIMD support in different x86 CPUs, SimSIMD generally uses the names of select Intel CPU generations for its backends.
  113. They, however, also work on AMD CPUs.
  114. Intel Haswell is compatible with AMD Zen 1/2/3, while AMD Genoa Zen 4 covers AVX-512 instructions added to Intel Skylake and Ice Lake.
  115. You can learn more about the technical implementation details in the following blog-posts:
  116. - [Uses Horner's method for polynomial approximations, beating GCC 12 by 119x](https://ashvardanian.com/posts/gcc-12-vs-avx512fp16/).
  117. - [Uses Arm SVE and x86 AVX-512's masked loads to eliminate tail `for`-loops](https://ashvardanian.com/posts/simsimd-faster-scipy/#tails-of-the-past-the-significance-of-masked-loads).
  118. - [Substitutes libc's `sqrt` with Newton Raphson iterations](https://github.com/ashvardanian/SimSIMD/releases/tag/v5.4.0).
  119. - [Uses Galloping and SVE2 histograms to intersect sparse vectors](https://ashvardanian.com/posts/simd-set-intersections-sve2-avx512/).
  120. - For Python: [avoids slow PyBind11, SWIG, & `PyArg_ParseTuple`](https://ashvardanian.com/posts/pybind11-cpython-tutorial/) [using faster calling convention](https://ashvardanian.com/posts/discount-on-keyword-arguments-in-python/).
  121. - For JavaScript: [uses typed arrays and NAPI for zero-copy calls](https://ashvardanian.com/posts/javascript-ai-vector-search/).
  122. ## Benchmarks
  123. <table style="width: 100%; text-align: center; table-layout: fixed;">
  124. <colgroup>
  125. <col style="width: 33%;">
  126. <col style="width: 33%;">
  127. <col style="width: 33%;">
  128. </colgroup>
  129. <tr>
  130. <th align="center">NumPy</th>
  131. <th align="center">C 99</th>
  132. <th align="center">SimSIMD</th>
  133. </tr>
  134. <!-- Cosine distances with different precision levels -->
  135. <tr>
  136. <td colspan="4" align="center">cosine distances between 1536d vectors in <code>int8</code></td>
  137. </tr>
  138. <tr>
  139. <td align="center"> <!-- scipy.spatial.distance.cosine -->
  140. 🚧 overflows<br/>
  141. </td>
  142. <td align="center"> <!-- serial -->
  143. <span style="color:#ABABAB;">x86:</span> <b>10,548,600</b> ops/s<br/>
  144. <span style="color:#ABABAB;">arm:</span> <b>11,379,300</b> ops/s
  145. </td>
  146. <td align="center"> <!-- simsimd -->
  147. <span style="color:#ABABAB;">x86:</span> <b>16,151,800</b> ops/s<br/>
  148. <span style="color:#ABABAB;">arm:</span> <b>13,524,000</b> ops/s
  149. </td>
  150. </tr>
  151. <tr>
  152. <td colspan="4" align="center">cosine distances between 1536d vectors in <code>bfloat16</code></td>
  153. </tr>
  154. <tr>
  155. <td align="center"> <!-- scipy.spatial.distance.cosine -->
  156. 🚧 not supported<br/>
  157. </td>
  158. <td align="center"> <!-- serial -->
  159. <span style="color:#ABABAB;">x86:</span> <b>119,835</b> ops/s<br/>
  160. <span style="color:#ABABAB;">arm:</span> <b>403,909</b> ops/s
  161. </td>
  162. <td align="center"> <!-- simsimd -->
  163. <span style="color:#ABABAB;">x86:</span> <b>9,738,540</b> ops/s<br/>
  164. <span style="color:#ABABAB;">arm:</span> <b>4,881,900</b> ops/s
  165. </td>
  166. </tr>
  167. <tr>
  168. <td colspan="4" align="center">cosine distances between 1536d vectors in <code>float16</code></td>
  169. </tr>
  170. <tr>
  171. <td align="center"> <!-- scipy.spatial.distance.cosine -->
  172. <span style="color:#ABABAB;">x86:</span> <b>40,481</b> ops/s<br/>
  173. <span style="color:#ABABAB;">arm:</span> <b>21,451</b> ops/s
  174. </td>
  175. <td align="center"> <!-- serial -->
  176. <span style="color:#ABABAB;">x86:</span> <b>501,310</b> ops/s<br/>
  177. <span style="color:#ABABAB;">arm:</span> <b>871,963</b> ops/s
  178. </td>
  179. <td align="center"> <!-- simsimd -->
  180. <span style="color:#ABABAB;">x86:</span> <b>7,627,600</b> ops/s<br/>
  181. <span style="color:#ABABAB;">arm:</span> <b>3,316,810</b> ops/s
  182. </td>
  183. </tr>
  184. <tr>
  185. <td colspan="4" align="center">cosine distances between 1536d vectors in <code>float32</code></td>
  186. </tr>
  187. <tr>
  188. <td align="center"> <!-- scipy.spatial.distance.cosine -->
  189. <span style="color:#ABABAB;">x86:</span> <b>253,902</b> ops/s<br/>
  190. <span style="color:#ABABAB;">arm:</span> <b>46,394</b> ops/s
  191. </td>
  192. <td align="center"> <!-- serial -->
  193. <span style="color:#ABABAB;">x86:</span> <b>882,484</b> ops/s<br/>
  194. <span style="color:#ABABAB;">arm:</span> <b>399,661</b> ops/s
  195. </td>
  196. <td align="center"> <!-- simsimd -->
  197. <span style="color:#ABABAB;">x86:</span> <b>8,202,910</b> ops/s<br/>
  198. <span style="color:#ABABAB;">arm:</span> <b>3,400,620</b> ops/s
  199. </td>
  200. </tr>
  201. <tr>
  202. <td colspan="4" align="center">cosine distances between 1536d vectors in <code>float64</code></td>
  203. </tr>
  204. <tr>
  205. <td align="center"> <!-- scipy.spatial.distance.cosine -->
  206. <span style="color:#ABABAB;">x86:</span> <b>212,421</b> ops/s<br/>
  207. <span style="color:#ABABAB;">arm:</span> <b>52,904</b> ops/s
  208. </td>
  209. <td align="center"> <!-- serial -->
  210. <span style="color:#ABABAB;">x86:</span> <b>839,301</b> ops/s<br/>
  211. <span style="color:#ABABAB;">arm:</span> <b>837,126</b> ops/s
  212. </td>
  213. <td align="center"> <!-- simsimd -->
  214. <span style="color:#ABABAB;">x86:</span> <b>1,538,530</b> ops/s<br/>
  215. <span style="color:#ABABAB;">arm:</span> <b>1,678,920</b> ops/s
  216. </td>
  217. </tr>
  218. <!-- Euclidean distance with different precision level -->
  219. <tr>
  220. <td colspan="4" align="center">euclidean distance between 1536d vectors in <code>int8</code></td>
  221. </tr>
  222. <tr>
  223. <td align="center"> <!-- scipy.spatial.distance.sqeuclidean -->
  224. <span style="color:#ABABAB;">x86:</span> <b>252,113</b> ops/s<br/>
  225. <span style="color:#ABABAB;">arm:</span> <b>177,443</b> ops/s
  226. </td>
  227. <td align="center"> <!-- serial -->
  228. <span style="color:#ABABAB;">x86:</span> <b>6,690,110</b> ops/s<br/>
  229. <span style="color:#ABABAB;">arm:</span> <b>4,114,160</b> ops/s
  230. </td>
  231. <td align="center"> <!-- simsimd -->
  232. <span style="color:#ABABAB;">x86:</span> <b>18,989,000</b> ops/s<br/>
  233. <span style="color:#ABABAB;">arm:</span> <b>18,878,200</b> ops/s
  234. </td>
  235. </tr>
  236. <tr>
  237. <td colspan="4" align="center">euclidean distance between 1536d vectors in <code>bfloat16</code></td>
  238. </tr>
  239. <tr>
  240. <td align="center"> <!-- scipy.spatial.distance.sqeuclidean -->
  241. 🚧 not supported<br/>
  242. </td>
  243. <td align="center"> <!-- serial -->
  244. <span style="color:#ABABAB;">x86:</span> <b>119,842</b> ops/s<br/>
  245. <span style="color:#ABABAB;">arm:</span> <b>1,049,230</b> ops/s
  246. </td>
  247. <td align="center"> <!-- simsimd -->
  248. <span style="color:#ABABAB;">x86:</span> <b>9,727,210</b> ops/s<br/>
  249. <span style="color:#ABABAB;">arm:</span> <b>4,233,420</b> ops/s
  250. </td>
  251. </tr>
  252. <tr>
  253. <td colspan="4" align="center">euclidean distance between 1536d vectors in <code>float16</code></td>
  254. </tr>
  255. <tr>
  256. <td align="center"> <!-- scipy.spatial.distance.sqeuclidean -->
  257. <span style="color:#ABABAB;">x86:</span> <b>54,621</b> ops/s<br/>
  258. <span style="color:#ABABAB;">arm:</span> <b>71,793</b> ops/s
  259. </td>
  260. <td align="center"> <!-- serial -->
  261. <span style="color:#ABABAB;">x86:</span> <b>196,413</b> ops/s<br/>
  262. <span style="color:#ABABAB;">arm:</span> <b>911,370</b> ops/s
  263. </td>
  264. <td align="center"> <!-- simsimd -->
  265. <span style="color:#ABABAB;">x86:</span> <b>19,466,800</b> ops/s<br/>
  266. <span style="color:#ABABAB;">arm:</span> <b>3,522,760</b> ops/s
  267. </td>
  268. </tr>
  269. <tr>
  270. <td colspan="4" align="center">euclidean distance between 1536d vectors in <code>float32</code></td>
  271. </tr>
  272. <tr>
  273. <td align="center"> <!-- scipy.spatial.distance.sqeuclidean -->
  274. <span style="color:#ABABAB;">x86:</span> <b>424,944</b> ops/s<br/>
  275. <span style="color:#ABABAB;">arm:</span> <b>292,629</b> ops/s
  276. </td>
  277. <td align="center"> <!-- serial -->
  278. <span style="color:#ABABAB;">x86:</span> <b>1,295,210</b> ops/s<br/>
  279. <span style="color:#ABABAB;">arm:</span> <b>1,055,940</b> ops/s
  280. </td>
  281. <td align="center"> <!-- simsimd -->
  282. <span style="color:#ABABAB;">x86:</span> <b>8,924,100</b> ops/s<br/>
  283. <span style="color:#ABABAB;">arm:</span> <b>3,602,650</b> ops/s
  284. </td>
  285. </tr>
  286. <tr>
  287. <td colspan="4" align="center">euclidean distance between 1536d vectors in <code>float64</code></td>
  288. </tr>
  289. <tr>
  290. <td align="center"> <!-- scipy.spatial.distance.sqeuclidean -->
  291. <span style="color:#ABABAB;">x86:</span> <b>334,929</b> ops/s<br/>
  292. <span style="color:#ABABAB;">arm:</span> <b>237,505</b> ops/s
  293. </td>
  294. <td align="center"> <!-- serial -->
  295. <span style="color:#ABABAB;">x86:</span> <b>1,215,190</b> ops/s<br/>
  296. <span style="color:#ABABAB;">arm:</span> <b>905,782</b> ops/s
  297. </td>
  298. <td align="center"> <!-- simsimd -->
  299. <span style="color:#ABABAB;">x86:</span> <b>1,701,740</b> ops/s<br/>
  300. <span style="color:#ABABAB;">arm:</span> <b>1,735,840</b> ops/s
  301. </td>
  302. </tr>
  303. <!-- Bilinear forms -->
  304. <!-- Sparse set intersections -->
  305. </table>
  306. > For benchmarks we mostly use 1536-dimensional vectors, like the embeddings produced by the OpenAI Ada API.
  307. > The code was compiled with GCC 12, using glibc v2.35.
  308. > The benchmarks performed on Arm-based Graviton3 AWS `c7g` instances and `r7iz` Intel Sapphire Rapids.
  309. > Most modern Arm-based 64-bit CPUs will have similar relative speedups.
  310. > Variance within x86 CPUs will be larger.
  311. Similar speedups are often observed even when compared to BLAS and LAPACK libraries underlying most numerical computing libraries, including NumPy and SciPy in Python.
  312. Broader benchmarking results:
  313. - [Apple M2 Pro](https://ashvardanian.com/posts/simsimd-faster-scipy/#appendix-1-performance-on-apple-m2-pro).
  314. - [Intel Sapphire Rapids](https://ashvardanian.com/posts/simsimd-faster-scipy/#appendix-2-performance-on-4th-gen-intel-xeon-platinum-8480).
  315. - [AWS Graviton 3](https://ashvardanian.com/posts/simsimd-faster-scipy/#appendix-3-performance-on-aws-graviton-3).
  316. ## Using SimSIMD in Python
  317. The package is intended to replace the usage of `numpy.inner`, `numpy.dot`, and `scipy.spatial.distance`.
  318. Aside from drastic performance improvements, SimSIMD significantly improves accuracy in mixed precision setups.
  319. NumPy and SciPy, processing `int8`, `uint8` or `float16` vectors, will use the same types for accumulators, while SimSIMD can combine `int8` enumeration, `int16` multiplication, and `int32` accumulation to avoid overflows entirely.
  320. The same applies to processing `float16` and `bfloat16` values with `float32` precision.
  321. ### Installation
  322. Use the following snippet to install SimSIMD and list hardware acceleration options available on your machine:
  323. ```sh
  324. pip install simsimd
  325. python -c "import simsimd; print(simsimd.get_capabilities())" # for hardware introspection
  326. python -c "import simsimd; help(simsimd)" # for documentation
  327. ```
  328. With precompiled binaries, SimSIMD ships `.pyi` interface files for type hinting and static analysis.
  329. You can check all the available functions in [`python/annotations/__init__.pyi`](https://github.com/ashvardanian/SimSIMD/blob/main/python/annotations/__init__.pyi).
  330. ### One-to-One Distance
  331. ```py
  332. import simsimd
  333. import numpy as np
  334. vec1 = np.random.randn(1536).astype(np.float32)
  335. vec2 = np.random.randn(1536).astype(np.float32)
  336. dist = simsimd.cosine(vec1, vec2)
  337. ```
  338. Supported functions include `cosine`, `inner`, `sqeuclidean`, `hamming`, `jaccard`, `kullbackleibler`, `jensenshannon`, and `intersect`.
  339. Dot products are supported for both real and complex numbers:
  340. ```py
  341. vec1 = np.random.randn(768).astype(np.float64) + 1j * np.random.randn(768).astype(np.float64)
  342. vec2 = np.random.randn(768).astype(np.float64) + 1j * np.random.randn(768).astype(np.float64)
  343. dist = simsimd.dot(vec1.astype(np.complex128), vec2.astype(np.complex128))
  344. dist = simsimd.dot(vec1.astype(np.complex64), vec2.astype(np.complex64))
  345. dist = simsimd.vdot(vec1.astype(np.complex64), vec2.astype(np.complex64)) # conjugate, same as `np.vdot`
  346. ```
  347. Unlike SciPy, SimSIMD allows explicitly stating the precision of the input vectors, which is especially useful for mixed-precision setups.
  348. The `dtype` argument can be passed both by name and as a positional argument:
  349. ```py
  350. dist = simsimd.cosine(vec1, vec2, "int8")
  351. dist = simsimd.cosine(vec1, vec2, "float16")
  352. dist = simsimd.cosine(vec1, vec2, "float32")
  353. dist = simsimd.cosine(vec1, vec2, "float64")
  354. dist = simsimd.hamming(vec1, vec2, "bin8")
  355. ```
  356. Binary distance functions are computed at a bit-level.
  357. Meaning a vector of 10x 8-bit integers will be treated as a sequence of 80 individual bits or dimensions.
  358. This differs from NumPy, that can't handle smaller-than-byte types, but you can still avoid the `bin8` argument by reinterpreting the vector as booleans:
  359. ```py
  360. vec1 = np.random.randint(2, size=80).astype(np.uint8).packbits().view(np.bool_)
  361. vec2 = np.random.randint(2, size=80).astype(np.uint8).packbits().view(np.bool_)
  362. hamming_distance = simsimd.hamming(vec1, vec2)
  363. jaccard_distance = simsimd.jaccard(vec1, vec2)
  364. ```
  365. With other frameworks, like PyTorch, one can get a richer type-system than NumPy, but the lack of good CPython interoperability makes it hard to pass data without copies.
  366. Here is an example of using SimSIMD with PyTorch to compute the cosine similarity between two `bfloat16` vectors:
  367. ```py
  368. import numpy as np
  369. buf1 = np.empty(8, dtype=np.uint16)
  370. buf2 = np.empty(8, dtype=np.uint16)
  371. # View the same memory region with PyTorch and randomize it
  372. import torch
  373. vec1 = torch.asarray(memoryview(buf1), copy=False).view(torch.bfloat16)
  374. vec2 = torch.asarray(memoryview(buf2), copy=False).view(torch.bfloat16)
  375. torch.randn(8, out=vec1)
  376. torch.randn(8, out=vec2)
  377. # Both libs will look into the same memory buffers and report the same results
  378. dist_slow = 1 - torch.nn.functional.cosine_similarity(vec1, vec2, dim=0)
  379. dist_fast = simsimd.cosine(buf1, buf2, "bfloat16")
  380. ```
  381. It also allows using SimSIMD for half-precision complex numbers, which NumPy does not support.
  382. For that, view data as continuous even-length `np.float16` vectors and override type-resolution with `complex32` string.
  383. ```py
  384. vec1 = np.random.randn(1536).astype(np.float16)
  385. vec2 = np.random.randn(1536).astype(np.float16)
  386. simd.dot(vec1, vec2, "complex32")
  387. simd.vdot(vec1, vec2, "complex32")
  388. ```
  389. When dealing with sparse representations and integer sets, you can apply the `intersect` function to two 1-dimensional arrays of `uint16` or `uint32` integers:
  390. ```py
  391. from random import randint
  392. import numpy as np
  393. import simsimd as simd
  394. length1, length2 = randint(1, 100), randint(1, 100)
  395. vec1 = np.sort(np.random.randint(0, 1000, length1).astype(np.uint16))
  396. vec2 = np.sort(np.random.randint(0, 1000, length2).astype(np.uint16))
  397. slow_result = len(np.intersect1d(vec1, vec2))
  398. fast_result = simd.intersect(vec1, vec2)
  399. assert slow_result == fast_result
  400. ```
  401. ### One-to-Many Distances
  402. Every distance function can be used not only for one-to-one but also one-to-many and many-to-many distance calculations.
  403. For one-to-many:
  404. ```py
  405. vec1 = np.random.randn(1536).astype(np.float32) # rank 1 tensor
  406. batch1 = np.random.randn(1, 1536).astype(np.float32) # rank 2 tensor
  407. batch2 = np.random.randn(100, 1536).astype(np.float32)
  408. dist_rank1 = simsimd.cosine(vec1, batch2)
  409. dist_rank2 = simsimd.cosine(batch1, batch2)
  410. ```
  411. ### Many-to-Many Distances
  412. All distance functions in SimSIMD can be used to compute many-to-many distances.
  413. For two batches of 100 vectors to compute 100 distances, one would call it like this:
  414. ```py
  415. batch1 = np.random.randn(100, 1536).astype(np.float32)
  416. batch2 = np.random.randn(100, 1536).astype(np.float32)
  417. dist = simsimd.cosine(batch1, batch2)
  418. ```
  419. Input matrices must have identical shapes.
  420. This functionality isn't natively present in NumPy or SciPy, and generally requires creating intermediate arrays, which is inefficient and memory-consuming.
  421. ### Many-to-Many All-Pairs Distances
  422. One can use SimSIMD to compute distances between all possible pairs of rows across two matrices (akin to [`scipy.spatial.distance.cdist`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.cdist.html)).
  423. The resulting object will have a type `DistancesTensor`, zero-copy compatible with NumPy and other libraries.
  424. For two arrays of 10 and 1,000 entries, the resulting tensor will have 10,000 cells:
  425. ```py
  426. import numpy as np
  427. from simsimd import cdist, DistancesTensor
  428. matrix1 = np.random.randn(1000, 1536).astype(np.float32)
  429. matrix2 = np.random.randn(10, 1536).astype(np.float32)
  430. distances: DistancesTensor = simsimd.cdist(matrix1, matrix2, metric="cosine") # zero-copy, managed by SimSIMD
  431. distances_array: np.ndarray = np.array(distances, copy=True) # now managed by NumPy
  432. ```
  433. ### Element-wise Kernels
  434. SimSIMD also provides mixed-precision element-wise kernels, where the input vectors and the output have the same numeric type, but the intermediate accumulators are of a higher precision.
  435. ```py
  436. import numpy as np
  437. from simsimd import fma, wsum
  438. # Let's take two FullHD video frames
  439. first_frame = np.random.randn(1920 * 1024).astype(np.uint8)
  440. second_frame = np.random.randn(1920 * 1024).astype(np.uint8)
  441. average_frame = np.empty_like(first_frame)
  442. wsum(first_frame, second_frame, alpha=0.5, beta=0.5, out=average_frame)
  443. # Slow analog with NumPy:
  444. slow_average_frame = (0.5 * first_frame + 0.5 * second_frame).astype(np.uint8)
  445. ```
  446. Similarly, the `fma` takes three arguments and computes the fused multiply-add operation.
  447. In applications like Machine Learning you may also benefit from using the "brain-float" format not natively supported by NumPy.
  448. In 3D Graphics, for example, we can use FMA to compute the [Phong shading model](https://en.wikipedia.org/wiki/Phong_shading):
  449. ```py
  450. # Assume a FullHD frame with random values for simplicity
  451. light_intensity = np.random.rand(1920 * 1080).astype(np.float16) # Intensity of light on each pixel
  452. diffuse_component = np.random.rand(1920 * 1080).astype(np.float16) # Diffuse reflectance on the surface
  453. specular_component = np.random.rand(1920 * 1080).astype(np.float16) # Specular reflectance for highlights
  454. output_color = np.empty_like(light_intensity) # Array to store the resulting color intensity
  455. # Define the scaling factors for diffuse and specular contributions
  456. alpha = 0.7 # Weight for the diffuse component
  457. beta = 0.3 # Weight for the specular component
  458. # Formula: color = alpha * light_intensity * diffuse_component + beta * specular_component
  459. fma(light_intensity, diffuse_component, specular_component,
  460. dtype="float16", # Optional, unless it can't be inferred from the input
  461. alpha=alpha, beta=beta, out=output_color)
  462. # Slow analog with NumPy for comparison
  463. slow_output_color = (alpha * light_intensity * diffuse_component + beta * specular_component).astype(np.float16)
  464. ```
  465. ### Multithreading and Memory Usage
  466. By default, computations use a single CPU core.
  467. To override this behavior, use the `threads` argument.
  468. Set it to `0` to use all available CPU cores and let the underlying C library manage the thread pool.
  469. Here is an example of dealing with large sets of binary vectors:
  470. ```py
  471. ndim = 1536 # OpenAI Ada embeddings
  472. matrix1 = np.packbits(np.random.randint(2, size=(10_000, ndim)).astype(np.uint8))
  473. matrix2 = np.packbits(np.random.randint(2, size=(1_000, ndim)).astype(np.uint8))
  474. distances = simsimd.cdist(matrix1, matrix2,
  475. metric="hamming", # Unlike SciPy, SimSIMD doesn't divide by the number of dimensions
  476. out_dtype="uint8", # so we can use `uint8` instead of `float64` to save memory.
  477. threads=0, # Use all CPU cores with OpenMP.
  478. dtype="bin8", # Override input argument type to `bin8` eight-bit words.
  479. )
  480. ```
  481. Alternatively, when using free-threading Python 3.13t builds, one can combine single-threaded SimSIMD operations with Python's `concurrent.futures.ThreadPoolExecutor` to parallelize the computations.
  482. By default, the output distances will be stored in double-precision `float64` floating-point numbers.
  483. That behavior may not be space-efficient, especially if you are computing the hamming distance between short binary vectors, that will generally fit into 8x smaller `uint8` or `uint16` types.
  484. To override this behavior, use the `out_dtype` argument, or consider pre-allocating the output array and passing it to the `out` argument.
  485. A more complete example may look like this:
  486. ```py
  487. from multiprocessing import cpu_count
  488. from concurrent.futures import ThreadPoolExecutor
  489. from simsimd import cosine
  490. import numpy as np
  491. # Generate large dataset
  492. vectors_a = np.random.rand(100_000, 1536).astype(np.float32)
  493. vectors_b = np.random.rand(100_000, 1536).astype(np.float32)
  494. distances = np.zeros((100_000,), dtype=np.float32)
  495. def compute_batch(start_idx, end_idx):
  496. batch_a = vectors_a[start_idx:end_idx]
  497. batch_b = vectors_b[start_idx:end_idx]
  498. cosine(batch_a, batch_b, out=distances[start_idx:end_idx])
  499. # Use all CPU cores with true parallelism (no GIL!)
  500. num_threads = cpu_count()
  501. chunk_size = len(vectors_a) // num_threads
  502. with ThreadPoolExecutor(max_workers=num_threads) as executor:
  503. futures = []
  504. for i in range(num_threads):
  505. start_idx = i * chunk_size
  506. end_idx = (i + 1) * chunk_size if i < num_threads - 1 else len(vectors_a)
  507. futures.append(executor.submit(compute_batch, start_idx, end_idx))
  508. # Collect results from all threads
  509. results = [future.result() for future in futures]
  510. ```
  511. ### Half-Precision Brain-Float Numbers
  512. The "brain-float-16" is a popular machine learning format.
  513. It's broadly supported in hardware and is very machine-friendly, but software support is still lagging behind.
  514. [Unlike NumPy](https://github.com/numpy/numpy/issues/19808), you can already use `bf16` datatype in SimSIMD.
  515. Luckily, to downcast `f32` to `bf16` you only have to drop the last 16 bits:
  516. ```py
  517. import numpy as np
  518. import simsimd as simd
  519. ndim = 1536
  520. a = np.random.randn(ndim).astype(np.float32)
  521. b = np.random.randn(ndim).astype(np.float32)
  522. # NumPy doesn't natively support brain-float, so we need a trick!
  523. # Luckily, it's very easy to reduce the representation accuracy
  524. # by simply masking the low 16-bits of our 32-bit single-precision
  525. # numbers. We can also add `0x8000` to round the numbers.
  526. a_f32rounded = ((a.view(np.uint32) + 0x8000) & 0xFFFF0000).view(np.float32)
  527. b_f32rounded = ((b.view(np.uint32) + 0x8000) & 0xFFFF0000).view(np.float32)
  528. # To represent them as brain-floats, we need to drop the second half
  529. a_bf16 = np.right_shift(a_f32rounded.view(np.uint32), 16).astype(np.uint16)
  530. b_bf16 = np.right_shift(b_f32rounded.view(np.uint32), 16).astype(np.uint16)
  531. # Now we can compare the results
  532. expected = np.inner(a_f32rounded, b_f32rounded)
  533. result = simd.inner(a_bf16, b_bf16, "bf16")
  534. ```
  535. ### Helper Functions
  536. You can turn specific backends on or off depending on the exact environment.
  537. A common case may be avoiding AVX-512 on older AMD CPUs and [Intel Ice Lake](https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html) CPUs to ensure the CPU doesn't change the frequency license and throttle performance.
  538. ```py
  539. $ simsimd.get_capabilities()
  540. > {'serial': True, 'neon': False, 'sve': False, 'neon_f16': False, 'sve_f16': False, 'neon_bf16': False, 'sve_bf16': False, 'neon_i8': False, 'sve_i8': False, 'haswell': True, 'skylake': True, 'ice': True, 'genoa': True, 'sapphire': True, 'turin': True}
  541. $ simsimd.disable_capability("sapphire")
  542. $ simsimd.enable_capability("sapphire")
  543. ```
  544. ### Using Python API with USearch
  545. Want to use it in Python with [USearch](https://github.com/unum-cloud/usearch)?
  546. You can wrap the raw C function pointers SimSIMD backends into a `CompiledMetric` and pass it to USearch, similar to how it handles Numba's JIT-compiled code.
  547. ```py
  548. from usearch.index import Index, CompiledMetric, MetricKind, MetricSignature
  549. from simsimd import pointer_to_sqeuclidean, pointer_to_cosine, pointer_to_inner
  550. metric = CompiledMetric(
  551. pointer=pointer_to_cosine("f16"),
  552. kind=MetricKind.Cos,
  553. signature=MetricSignature.ArrayArraySize,
  554. )
  555. index = Index(256, metric=metric)
  556. ```
  557. ## Using SimSIMD in Rust
  558. To install, add the following to your `Cargo.toml`:
  559. ```toml
  560. [dependencies]
  561. simsimd = "..."
  562. ```
  563. Before using the SimSIMD library, ensure you have imported the necessary traits and types into your Rust source file.
  564. The library provides several traits for different distance/similarity kinds - `SpatialSimilarity`, `BinarySimilarity`, and `ProbabilitySimilarity`.
  565. ### Spatial Similarity: Cosine and Euclidean Distances
  566. ```rust
  567. use simsimd::SpatialSimilarity;
  568. fn main() {
  569. let vector_a: Vec<f32> = vec![1.0, 2.0, 3.0];
  570. let vector_b: Vec<f32> = vec![4.0, 5.0, 6.0];
  571. // Compute the cosine distance between vectors
  572. let cosine_distance = f32::cosine(&vector_a, &vector_b)
  573. .expect("Vectors must be of the same length");
  574. println!("Cosine Distance: {}", cosine_distance);
  575. // Compute the squared Euclidean distance between vectors
  576. let sq_euclidean_distance = f32::sqeuclidean(&vector_a, &vector_b)
  577. .expect("Vectors must be of the same length");
  578. println!("Squared Euclidean Distance: {}", sq_euclidean_distance);
  579. }
  580. ```
  581. Spatial similarity functions are available for `f64`, `f32`, `f16`, and `i8` types.
  582. ### Dot-Products: Inner and Complex Inner Products
  583. ```rust
  584. use simsimd::SpatialSimilarity;
  585. use simsimd::ComplexProducts;
  586. fn main() {
  587. // Complex vectors have interleaved real & imaginary components
  588. let vector_a: Vec<f32> = vec![1.0, 2.0, 3.0, 4.0];
  589. let vector_b: Vec<f32> = vec![5.0, 6.0, 7.0, 8.0];
  590. // Compute the inner product between vectors
  591. let inner_product = SpatialSimilarity::dot(&vector_a, &vector_b)
  592. .expect("Vectors must be of the same length");
  593. println!("Inner Product: {}", inner_product);
  594. // Compute the complex inner product between vectors
  595. let complex_inner_product = ComplexProducts::dot(&vector_a, &vector_b)
  596. .expect("Vectors must be of the same length");
  597. let complex_conjugate_inner_product = ComplexProducts::vdot(&vector_a, &vector_b)
  598. .expect("Vectors must be of the same length");
  599. println!("Complex Inner Product: {:?}", complex_inner_product); // -18, 69
  600. println!("Complex C. Inner Product: {:?}", complex_conjugate_inner_product); // 70, -8
  601. }
  602. ```
  603. Complex inner products are available for `f64`, `f32`, and `f16` types.
  604. ### Probability Distributions: Jensen-Shannon and Kullback-Leibler Divergences
  605. ```rust
  606. use simsimd::ProbabilitySimilarity;
  607. fn main() {
  608. let vector_a: Vec<f32> = vec![1.0, 2.0, 3.0];
  609. let vector_b: Vec<f32> = vec![4.0, 5.0, 6.0];
  610. let jensen_shannon = f32::jensenshannon(&vector_a, &vector_b)
  611. .expect("Vectors must be of the same length");
  612. println!("Jensen-Shannon Divergence: {}", jensen_shannon);
  613. let kullback_leibler = f32::kullbackleibler(&vector_a, &vector_b)
  614. .expect("Vectors must be of the same length");
  615. println!("Kullback-Leibler Divergence: {}", kullback_leibler);
  616. }
  617. ```
  618. Probability similarity functions are available for `f64`, `f32`, and `f16` types.
  619. ### Binary Similarity: Hamming and Jaccard Distances
  620. Similar to spatial distances, one can compute bit-level distance functions between slices of unsigned integers:
  621. ```rust
  622. use simsimd::BinarySimilarity;
  623. fn main() {
  624. let vector_a = &[0b11110000, 0b00001111, 0b10101010];
  625. let vector_b = &[0b11110000, 0b00001111, 0b01010101];
  626. // Compute the Hamming distance between vectors
  627. let hamming_distance = u8::hamming(&vector_a, &vector_b)
  628. .expect("Vectors must be of the same length");
  629. println!("Hamming Distance: {}", hamming_distance);
  630. // Compute the Jaccard distance between vectors
  631. let jaccard_distance = u8::jaccard(&vector_a, &vector_b)
  632. .expect("Vectors must be of the same length");
  633. println!("Jaccard Distance: {}", jaccard_distance);
  634. }
  635. ```
  636. Binary similarity functions are available only for `u8` types.
  637. ### Half-Precision Floating-Point Numbers
  638. Rust has no native support for half-precision floating-point numbers, but SimSIMD provides a `f16` type with built-in conversion methods.
  639. The underlying `u16` representation is publicly accessible for direct bit manipulation.
  640. ```rust
  641. use simsimd::{SpatialSimilarity, f16};
  642. fn main() {
  643. // Create f16 vectors using built-in conversion methods
  644. let vector_a: Vec<f16> = vec![1.0, 2.0, 3.0].iter().map(|&x| f16::from_f32(x)).collect();
  645. let vector_b: Vec<f16> = vec![4.0, 5.0, 6.0].iter().map(|&x| f16::from_f32(x)).collect();
  646. // Compute the cosine distance
  647. let cosine_distance = f16::cosine(&vector_a, &vector_b)
  648. .expect("Vectors must be of the same length");
  649. println!("Cosine Distance: {}", cosine_distance);
  650. // Direct bit manipulation
  651. let half = f16::from_f32(3.14159);
  652. let bits = half.0; // Access raw u16 representation
  653. let reconstructed = f16(bits);
  654. // Convert back to f32
  655. let float_value = half.to_f32();
  656. }
  657. ```
  658. For interoperability with the `half` crate:
  659. ```rust
  660. use simsimd::{SpatialSimilarity, f16 as SimF16};
  661. use half::f16 as HalfF16;
  662. fn main() {
  663. let vector_a: Vec<HalfF16> = vec![1.0, 2.0, 3.0].iter().map(|&x| HalfF16::from_f32(x)).collect();
  664. let vector_b: Vec<HalfF16> = vec![4.0, 5.0, 6.0].iter().map(|&x| HalfF16::from_f32(x)).collect();
  665. // Safe reinterpret cast due to identical memory layout
  666. let buffer_a: &[SimF16] = unsafe { std::slice::from_raw_parts(vector_a.as_ptr() as *const SimF16, vector_a.len()) };
  667. let buffer_b: &[SimF16] = unsafe { std::slice::from_raw_parts(vector_b.as_ptr() as *const SimF16, vector_b.len()) };
  668. let cosine_distance = SimF16::cosine(buffer_a, buffer_b)
  669. .expect("Vectors must be of the same length");
  670. println!("Cosine Distance: {}", cosine_distance);
  671. }
  672. ```
  673. ### Half-Precision Brain-Float Numbers
  674. The "brain-float-16" is a popular machine learning format.
  675. It's broadly supported in hardware and is very machine-friendly, but software support is still lagging behind.
  676. [Unlike NumPy](https://github.com/numpy/numpy/issues/19808), you can already use `bf16` datatype in SimSIMD.
  677. SimSIMD provides a `bf16` type with built-in conversion methods and direct bit access.
  678. ```rust
  679. use simsimd::{SpatialSimilarity, bf16};
  680. fn main() {
  681. // Create bf16 vectors using built-in conversion methods
  682. let vector_a: Vec<bf16> = vec![1.0, 2.0, 3.0].iter().map(|&x| bf16::from_f32(x)).collect();
  683. let vector_b: Vec<bf16> = vec![4.0, 5.0, 6.0].iter().map(|&x| bf16::from_f32(x)).collect();
  684. // Compute the cosine similarity
  685. let cosine_distance = bf16::cosine(&vector_a, &vector_b)
  686. .expect("Vectors must be of the same length");
  687. println!("Cosine Distance: {}", cosine_distance);
  688. // Direct bit manipulation
  689. let brain_half = bf16::from_f32(3.14159);
  690. let bits = brain_half.0; // Access raw u16 representation
  691. let reconstructed = bf16(bits);
  692. // Convert back to f32
  693. let float_value = brain_half.to_f32();
  694. // Compare precision differences
  695. let original = 3.14159_f32;
  696. let f16_roundtrip = f16::from_f32(original).to_f32();
  697. let bf16_roundtrip = bf16::from_f32(original).to_f32();
  698. println!("Original: {}", original);
  699. println!("f16 roundtrip: {}", f16_roundtrip);
  700. println!("bf16 roundtrip: {}", bf16_roundtrip);
  701. }
  702. ```
  703. ### Dynamic Dispatch in Rust
  704. SimSIMD provides a [dynamic dispatch](#dynamic-dispatch) mechanism to select the most advanced micro-kernel for the current CPU.
  705. You can query supported backends and use the `SimSIMD::capabilities` function to select the best one.
  706. ```rust
  707. println!("uses neon: {}", capabilities::uses_neon());
  708. println!("uses sve: {}", capabilities::uses_sve());
  709. println!("uses haswell: {}", capabilities::uses_haswell());
  710. println!("uses skylake: {}", capabilities::uses_skylake());
  711. println!("uses ice: {}", capabilities::uses_ice());
  712. println!("uses genoa: {}", capabilities::uses_genoa());
  713. println!("uses sapphire: {}", capabilities::uses_sapphire());
  714. println!("uses turin: {}", capabilities::uses_turin());
  715. println!("uses sierra: {}", capabilities::uses_sierra());
  716. ```
  717. ## Using SimSIMD in JavaScript
  718. To install, choose one of the following options depending on your environment:
  719. - `npm install --save simsimd`
  720. - `yarn add simsimd`
  721. - `pnpm add simsimd`
  722. - `bun install simsimd`
  723. The package is distributed with prebuilt binaries, but if your platform is not supported, you can build the package from the source via `npm run build`.
  724. This will automatically happen unless you install the package with the `--ignore-scripts` flag or use Bun.
  725. After you install it, you will be able to call the SimSIMD functions on various `TypedArray` variants:
  726. ```js
  727. const { sqeuclidean, cosine, inner, hamming, jaccard } = require("simsimd");
  728. const vectorA = new Float32Array([1.0, 2.0, 3.0]);
  729. const vectorB = new Float32Array([4.0, 5.0, 6.0]);
  730. const distance = sqeuclidean(vectorA, vectorB);
  731. console.log("Squared Euclidean Distance:", distance);
  732. ```
  733. Other numeric types and precision levels are supported as well.
  734. For double-precision floating-point numbers, use `Float64Array`:
  735. ```js
  736. const vectorA = new Float64Array([1.0, 2.0, 3.0]);
  737. const vectorB = new Float64Array([4.0, 5.0, 6.0]);
  738. const distance = cosine(vectorA, vectorB);
  739. ```
  740. When doing machine learning and vector search with high-dimensional vectors you may want to quantize them to 8-bit integers.
  741. You may want to project values from the $[-1, 1]$ range to the $[-127, 127]$ range and then cast them to `Int8Array`:
  742. ```js
  743. const quantizedVectorA = new Int8Array(vectorA.map((v) => v * 127));
  744. const quantizedVectorB = new Int8Array(vectorB.map((v) => v * 127));
  745. const distance = cosine(quantizedVectorA, quantizedVectorB);
  746. ```
  747. A more extreme quantization case would be to use binary vectors.
  748. You can map all positive values to `1` and all negative values and zero to `0`, packing eight values into a single byte.
  749. After that, Hamming and Jaccard distances can be computed.
  750. ```js
  751. const { toBinary, hamming } = require("simsimd");
  752. const binaryVectorA = toBinary(vectorA);
  753. const binaryVectorB = toBinary(vectorB);
  754. const distance = hamming(binaryVectorA, binaryVectorB);
  755. ```
  756. ## Using SimSIMD in Swift
  757. To install, simply add the following dependency to your `Package.swift`:
  758. ```swift
  759. dependencies: [
  760. .package(url: "https://github.com/ashvardanian/simsimd")
  761. ]
  762. ```
  763. The package provides the most common spatial metrics for `Int8`, `Float16`, `Float32`, and `Float64` vectors.
  764. ```swift
  765. import SimSIMD
  766. let vectorA: [Int8] = [1, 2, 3]
  767. let vectorB: [Int8] = [4, 5, 6]
  768. let dotProduct = vectorA.dot(vectorB) // Computes the dot product
  769. let cosineDistance = vectorA.cosine(vectorB) // Computes the cosine distance
  770. let sqEuclidean = vectorA.sqeuclidean(vectorB) // Computes the squared Euclidean distance
  771. ```
  772. ## Using SimSIMD in C
  773. For integration within a CMake-based project, add the following segment to your `CMakeLists.txt`:
  774. ```cmake
  775. FetchContent_Declare(
  776. simsimd
  777. GIT_REPOSITORY https://github.com/ashvardanian/simsimd.git
  778. GIT_SHALLOW TRUE
  779. )
  780. FetchContent_MakeAvailable(simsimd)
  781. ```
  782. After that, you can use the SimSIMD library in your C code in several ways.
  783. Simplest of all, you can include the headers, and the compiler will automatically select the most recent CPU extensions that SimSIMD will use.
  784. ```c
  785. #include <simsimd/simsimd.h>
  786. int main() {
  787. simsimd_flush_denormals(); // Optional, to avoid performance penalties on denormal numbers
  788. simsimd_f32_t vector_a[1536];
  789. simsimd_f32_t vector_b[1536];
  790. simsimd_kernel_punned_t metric_punned = simsimd_metric_punned(
  791. simsimd_metric_cos_k, // Metric kind, like the angular cosine distance
  792. simsimd_datatype_f32_k, // Data type, like: f16, f32, f64, i8, b8, and complex variants
  793. simsimd_cap_any_k); // Which CPU capabilities are we allowed to use
  794. simsimd_distance_t distance;
  795. simsimd_metric_dense_punned_t metric = (simsimd_metric_dense_punned_t)metric_punned;
  796. metric(vector_a, vector_b, 1536, &distance);
  797. return 0;
  798. }
  799. ```
  800. ### Dynamic Dispatch in C
  801. To avoid hard-coding the backend, you can rely on `c/lib.c` to prepackage all possible backends in one binary, and select the most recent CPU features at runtime.
  802. That feature of the C library is called [dynamic dispatch](#dynamic-dispatch) and is extensively used in the Python, JavaScript, and Rust bindings.
  803. To test which CPU features are available on the machine at runtime, use the following APIs:
  804. ```c
  805. int uses_dynamic_dispatch = simsimd_uses_dynamic_dispatch(); // Check if dynamic dispatch was enabled
  806. simsimd_capability_t capabilities = simsimd_capabilities(); // Returns a bitmask
  807. int uses_neon = simsimd_uses_neon();
  808. int uses_sve = simsimd_uses_sve();
  809. int uses_haswell = simsimd_uses_haswell();
  810. int uses_skylake = simsimd_uses_skylake();
  811. int uses_ice = simsimd_uses_ice();
  812. int uses_genoa = simsimd_uses_genoa();
  813. int uses_sapphire = simsimd_uses_sapphire();
  814. ```
  815. To override compilation settings and switch between runtime and compile-time dispatch, define the following macro:
  816. ```c
  817. #define SIMSIMD_DYNAMIC_DISPATCH 1 // or 0
  818. ```
  819. ### Spatial Distances: Cosine and Euclidean Distances
  820. ```c
  821. #include <simsimd/simsimd.h>
  822. int main() {
  823. simsimd_i8_t i8s[1536];
  824. simsimd_u8_t u8s[1536];
  825. simsimd_f64_t f64s[1536];
  826. simsimd_f32_t f32s[1536];
  827. simsimd_f16_t f16s[1536];
  828. simsimd_bf16_t bf16s[1536];
  829. simsimd_distance_t distance;
  830. // Cosine distance between two vectors
  831. simsimd_cos_i8(i8s, i8s, 1536, &distance);
  832. simsimd_cos_u8(u8s, u8s, 1536, &distance);
  833. simsimd_cos_f16(f16s, f16s, 1536, &distance);
  834. simsimd_cos_f32(f32s, f32s, 1536, &distance);
  835. simsimd_cos_f64(f64s, f64s, 1536, &distance);
  836. simsimd_cos_bf16(bf16s, bf16s, 1536, &distance);
  837. // Euclidean distance between two vectors
  838. simsimd_l2sq_i8(i8s, i8s, 1536, &distance);
  839. simsimd_l2sq_u8(u8s, u8s, 1536, &distance);
  840. simsimd_l2sq_f16(f16s, f16s, 1536, &distance);
  841. simsimd_l2sq_f32(f32s, f32s, 1536, &distance);
  842. simsimd_l2sq_f64(f64s, f64s, 1536, &distance);
  843. simsimd_l2sq_bf16(bf16s, bf16s, 1536, &distance);
  844. return 0;
  845. }
  846. ```
  847. ### Dot-Products: Inner and Complex Inner Products
  848. ```c
  849. #include <simsimd/simsimd.h>
  850. int main() {
  851. // SimSIMD provides "sized" type-aliases without relying on `stdint.h`
  852. simsimd_i8_t i8[1536];
  853. simsimd_i8_t u8[1536];
  854. simsimd_f16_t f16s[1536];
  855. simsimd_f32_t f32s[1536];
  856. simsimd_f64_t f64s[1536];
  857. simsimd_bf16_t bf16s[1536];
  858. simsimd_distance_t product;
  859. // Inner product between two real vectors
  860. simsimd_dot_i8(i8s, i8s, 1536, &product);
  861. simsimd_dot_u8(u8s, u8s, 1536, &product);
  862. simsimd_dot_f16(f16s, f16s, 1536, &product);
  863. simsimd_dot_f32(f32s, f32s, 1536, &product);
  864. simsimd_dot_f64(f64s, f64s, 1536, &product);
  865. simsimd_dot_bf16(bf16s, bf16s, 1536, &product);
  866. // SimSIMD provides complex types with `real` and `imag` fields
  867. simsimd_f64c_t f64cs[768];
  868. simsimd_f32c_t f32cs[768];
  869. simsimd_f16c_t f16cs[768];
  870. simsimd_bf16c_t bf16cs[768];
  871. simsimd_distance_t products[2]; // real and imaginary parts
  872. // Complex inner product between two vectors
  873. simsimd_dot_f16c(f16cs, f16cs, 768, &products[0]);
  874. simsimd_dot_f32c(f32cs, f32cs, 768, &products[0]);
  875. simsimd_dot_f64c(f64cs, f64cs, 768, &products[0]);
  876. simsimd_dot_bf16c(bf16cs, bf16cs, 768, &products[0]);
  877. // Complex conjugate inner product between two vectors
  878. simsimd_vdot_f16c(f16cs, f16cs, 768, &products[0]);
  879. simsimd_vdot_f32c(f32cs, f32cs, 768, &products[0]);
  880. simsimd_vdot_f64c(f64cs, f64cs, 768, &products[0]);
  881. simsimd_vdot_bf16c(bf16cs, bf16cs, 768, &products[0]);
  882. return 0;
  883. }
  884. ```
  885. ### Binary Distances: Hamming and Jaccard Distances
  886. ```c
  887. #include <simsimd/simsimd.h>
  888. int main() {
  889. simsimd_b8_t b8s[1536 / 8]; // 8 bits per word
  890. simsimd_distance_t distance;
  891. simsimd_hamming_b8(b8s, b8s, 1536 / 8, &distance);
  892. simsimd_jaccard_b8(b8s, b8s, 1536 / 8, &distance);
  893. return 0;
  894. }
  895. ```
  896. ### Probability Distributions: Jensen-Shannon and Kullback-Leibler Divergences
  897. ```c
  898. #include <simsimd/simsimd.h>
  899. int main() {
  900. simsimd_f64_t f64s[1536];
  901. simsimd_f32_t f32s[1536];
  902. simsimd_f16_t f16s[1536];
  903. simsimd_distance_t divergence;
  904. // Jensen-Shannon divergence between two vectors
  905. simsimd_js_f16(f16s, f16s, 1536, &divergence);
  906. simsimd_js_f32(f32s, f32s, 1536, &divergence);
  907. simsimd_js_f64(f64s, f64s, 1536, &divergence);
  908. // Kullback-Leibler divergence between two vectors
  909. simsimd_kl_f16(f16s, f16s, 1536, &divergence);
  910. simsimd_kl_f32(f32s, f32s, 1536, &divergence);
  911. simsimd_kl_f64(f64s, f64s, 1536, &divergence);
  912. return 0;
  913. }
  914. ```
  915. ### Half-Precision Floating-Point Numbers
  916. If you aim to utilize the `_Float16` functionality with SimSIMD, ensure your development environment is compatible with C 11.
  917. For other SimSIMD functionalities, C 99 compatibility will suffice.
  918. To explicitly disable half-precision support, define the following macro before imports:
  919. ```c
  920. #define SIMSIMD_NATIVE_F16 0 // or 1
  921. #define SIMSIMD_NATIVE_BF16 0 // or 1
  922. #include <simsimd/simsimd.h>
  923. ```
  924. ### Compilation Settings and Debugging
  925. `SIMSIMD_DYNAMIC_DISPATCH`:
  926. > By default, SimSIMD is a header-only library.
  927. > But if you are running on different generations of devices, it makes sense to pre-compile the library for all supported generations at once, and dispatch at runtime.
  928. > This flag does just that and is used to produce the `simsimd.so` shared library, as well as the Python and other bindings.
  929. For Arm: `SIMSIMD_TARGET_NEON`, `SIMSIMD_TARGET_SVE`, `SIMSIMD_TARGET_SVE2`, `SIMSIMD_TARGET_NEON_F16`, `SIMSIMD_TARGET_SVE_F16`, `SIMSIMD_TARGET_NEON_BF16`, `SIMSIMD_TARGET_SVE_BF16`.
  930. For x86: `SIMSIMD_TARGET_HASWELL`, `SIMSIMD_TARGET_SKYLAKE`, `SIMSIMD_TARGET_ICE`, `SIMSIMD_TARGET_GENOA`, `SIMSIMD_TARGET_SAPPHIRE`, `SIMSIMD_TARGET_TURIN`, `SIMSIMD_TARGET_SIERRA`.
  931. > By default, SimSIMD automatically infers the target architecture and pre-compiles as many kernels as possible.
  932. > In some cases, you may want to explicitly disable some of the kernels.
  933. > Most often it's due to compiler support issues, like the lack of some recent intrinsics or low-precision numeric types.
  934. > In other cases, you may want to disable some kernels to speed up the compilation process and trim the binary size.
  935. `SIMSIMD_SQRT`, `SIMSIMD_RSQRT`, `SIMSIMD_LOG`:
  936. > By default, for __non__-SIMD backends, SimSIMD may use `libc` functions like `sqrt` and `log`.
  937. > Those are generally very accurate, but slow, and introduce a dependency on the C standard library.
  938. > To avoid that you can override those definitions with your custom implementations, like: `#define SIMSIMD_RSQRT(x) (1 / sqrt(x))`.
  939. ## Algorithms & Design Decisions 📚
  940. In general there are a few principles that SimSIMD follows:
  941. - Avoid loop unrolling.
  942. - Never allocate memory.
  943. - Never throw exceptions or set `errno`.
  944. - Keep all function arguments the size of the pointer.
  945. - Avoid returning from public interfaces, use out-arguments instead.
  946. - Don't over-optimize for old CPUs and single- and double-precision floating-point numbers.
  947. - Prioritize mixed-precision and integer operations, and new ISA extensions.
  948. - Prefer saturated arithmetic and avoid overflows.
  949. Possibly, in the future:
  950. - Best effort computation silencing `NaN` components in low-precision inputs.
  951. - Detect overflows and report the distance with a "signaling" `NaN`.
  952. Last, but not the least - don't build unless there is a demand for it.
  953. So if you have a specific use-case, please open an issue or a pull request, and ideally, bring in more users with similar needs.
  954. ### Cosine Similarity, Reciprocal Square Root, and Newton-Raphson Iteration
  955. The cosine similarity is the most common and straightforward metric used in machine learning and information retrieval.
  956. Interestingly, there are multiple ways to shoot yourself in the foot when computing it.
  957. The cosine similarity is the inverse of the cosine distance, which is the cosine of the angle between two vectors.
  958. ```math
  959. \text{CosineSimilarity}(a, b) = \frac{a \cdot b}{\|a\| \cdot \|b\|}
  960. ```
  961. ```math
  962. \text{CosineDistance}(a, b) = 1 - \frac{a \cdot b}{\|a\| \cdot \|b\|}
  963. ```
  964. In NumPy terms, SimSIMD implementation is similar to:
  965. ```python
  966. import numpy as np
  967. def cos_numpy(a: np.ndarray, b: np.ndarray) -> float:
  968. ab, a2, b2 = np.dot(a, b), np.dot(a, a), np.dot(b, b) # Fused in SimSIMD
  969. if a2 == 0 and b2 == 0: result = 0 # Same in SciPy
  970. elif ab == 0: result = 1 # Division by zero error in SciPy
  971. else: result = 1 - ab / (sqrt(a2) * sqrt(b2)) # Bigger rounding error in SciPy
  972. return result
  973. ```
  974. In SciPy, however, the cosine distance is computed as `1 - ab / np.sqrt(a2 * b2)`.
  975. It handles the edge case of a zero and non-zero argument pair differently, resulting in a division by zero error.
  976. It's not only less efficient, but also less accurate, given how the reciprocal square roots are computed.
  977. The C standard library provides the `sqrt` function, which is generally very accurate, but slow.
  978. The `rsqrt` in-hardware implementations are faster, but have different accuracy characteristics.
  979. - SSE `rsqrtps` and AVX `vrsqrtps`: $1.5 \times 2^{-12}$ maximal relative error.
  980. - AVX-512 `vrsqrt14pd` instruction: $2^{-14}$ maximal relative error.
  981. - NEON `frsqrte` instruction has no documented error bounds, but [can be][arm-rsqrt] $2^{-3}$.
  982. [arm-rsqrt]: https://gist.github.com/ashvardanian/5e5cf585d63f8ab6d240932313c75411
  983. To overcome the limitations of the `rsqrt` instruction, SimSIMD uses the Newton-Raphson iteration to refine the initial estimate for high-precision floating-point numbers.
  984. It can be defined as:
  985. ```math
  986. x_{n+1} = x_n \cdot (3 - x_n \cdot x_n) / 2
  987. ```
  988. On 1536-dimensional inputs on Intel Sapphire Rapids CPU a single such iteration can result in a 2-3 orders of magnitude relative error reduction:
  989. | Datatype | NumPy Error | SimSIMD w/out Iteration | SimSIMD |
  990. | :--------- | ------------------: | ----------------------: | ------------------: |
  991. | `bfloat16` | 1.89e-08 ± 1.59e-08 | 3.07e-07 ± 3.09e-07 | 3.53e-09 ± 2.70e-09 |
  992. | `float16` | 1.67e-02 ± 1.44e-02 | 2.68e-05 ± 1.95e-05 | 2.02e-05 ± 1.39e-05 |
  993. | `float32` | 2.21e-08 ± 1.65e-08 | 3.47e-07 ± 3.49e-07 | 3.77e-09 ± 2.84e-09 |
  994. | `float64` | 0.00e+00 ± 0.00e+00 | 3.80e-07 ± 4.50e-07 | 1.35e-11 ± 1.85e-11 |
  995. ### Curved Spaces, Mahalanobis Distance, and Bilinear Quadratic Forms
  996. The Mahalanobis distance is a generalization of the Euclidean distance, which takes into account the covariance of the data.
  997. It's very similar in its form to the bilinear form, which is a generalization of the dot product.
  998. ```math
  999. \text{BilinearForm}(a, b, M) = a^T M b
  1000. ```
  1001. ```math
  1002. \text{Mahalanobis}(a, b, M) = \sqrt{(a - b)^T M^{-1} (a - b)}
  1003. ```
  1004. Bilinear Forms can be seen as one of the most important linear algebraic operations, surprisingly missing in BLAS and LAPACK.
  1005. They are versatile and appear in various domains:
  1006. - In Quantum Mechanics, the expectation value of an observable $A$ in a state $\psi$ is given by $\langle \psi | A | \psi \rangle$, which is a bilinear form.
  1007. - In Machine Learning, in Support Vector Machines (SVMs), bilinear forms define kernel functions that measure similarity between data points.
  1008. - In Differential Geometry, the metric tensor, which defines distances and angles on a manifold, is a bilinear form on the tangent space.
  1009. - In Economics, payoff functions in certain Game Theoretic problems can be modeled as bilinear forms of players' strategies.
  1010. - In Physics, interactions between electric and magnetic fields can be expressed using bilinear forms.
  1011. Broad applications aside, the lack of a specialized primitive for bilinear forms in BLAS and LAPACK means significant performance overhead.
  1012. A $vector * matrix * vector$ product is a scalar, whereas its constituent parts ($vector * matrix$ and $matrix * vector$) are vectors:
  1013. - They need memory to be stored in: $O(n)$ allocation.
  1014. - The data will be written to memory and read back, wasting CPU cycles.
  1015. SimSIMD doesn't produce intermediate vector results, like `a @ M @ b`, but computes the bilinear form directly.
  1016. ### Set Intersection, Galloping, and Binary Search
  1017. The set intersection operation is generally defined as the number of elements that are common between two sets, represented as sorted arrays of integers.
  1018. The most common way to compute it is a linear scan:
  1019. ```c
  1020. size_t intersection_size(int *a, int *b, size_t n, size_t m) {
  1021. size_t i = 0, j = 0, count = 0;
  1022. while (i < n && j < m) {
  1023. if (a[i] < b[j]) i++;
  1024. else if (a[i] > b[j]) j++;
  1025. else i++, j++, count++;
  1026. }
  1027. return count;
  1028. }
  1029. ```
  1030. Alternatively, one can use the binary search to find the elements in the second array that are present in the first one.
  1031. On every step the checked region of the second array is halved, which is called the _galloping search_.
  1032. It's faster, but only when large arrays of very different sizes are intersected.
  1033. Third approach is to use the SIMD instructions to compare multiple elements at once:
  1034. - Using string-intersection instructions on x86, like `pcmpestrm`.
  1035. - Using integer-intersection instructions in AVX-512, like `vp2intersectd`.
  1036. - Using vanilla equality checks present in all SIMD instruction sets.
  1037. After benchmarking, the last approach was chosen, as it's the most flexible and often the fastest.
  1038. ### Complex Dot Products, Conjugate Dot Products, and Complex Numbers
  1039. Complex dot products are a generalization of the dot product to complex numbers.
  1040. They are supported by most BLAS packages, but almost never in mixed precision.
  1041. SimSIMD defines `dot` and `vdot` kernels as:
  1042. ```math
  1043. \text{dot}(a, b) = \sum_{i=0}^{n-1} a_i \cdot b_i
  1044. ```
  1045. ```math
  1046. \text{vdot}(a, b) = \sum_{i=0}^{n-1} a_i \cdot \bar{b_i}
  1047. ```
  1048. Where $\bar{b_i}$ is the complex conjugate of $b_i$.
  1049. Putting that into Python code for scalar arrays:
  1050. ```python
  1051. def dot(a: List[number], b: List[number]) -> number:
  1052. a_real, a_imaginary = a[0::2], a[1::2]
  1053. b_real, b_imaginary = b[0::2], b[1::2]
  1054. ab_real, ab_imaginary = 0, 0
  1055. for ar, ai, br, bi in zip(a_real, a_imaginary, b_real, b_imaginary):
  1056. ab_real += ar * br - ai * bi
  1057. ab_imaginary += ar * bi + ai * br
  1058. return ab_real, ab_imaginary
  1059. def vdot(a: List[number], b: List[number]) -> number:
  1060. a_real, a_imaginary = a[0::2], a[1::2]
  1061. b_real, b_imaginary = b[0::2], b[1::2]
  1062. ab_real, ab_imaginary = 0, 0
  1063. for ar, ai, br, bi in zip(a_real, a_imaginary, b_real, b_imaginary):
  1064. ab_real += ar * br + ai * bi
  1065. ab_imaginary += ar * bi - ai * br
  1066. return ab_real, ab_imaginary
  1067. ```
  1068. ### Logarithms in Kullback-Leibler & Jensen–Shannon Divergences
  1069. The Kullback-Leibler divergence is a measure of how one probability distribution diverges from a second, expected probability distribution.
  1070. Jensen-Shannon divergence is a symmetrized and smoothed version of the Kullback-Leibler divergence, which can be used as a distance metric between probability distributions.
  1071. ```math
  1072. \text{KL}(P || Q) = \sum_{i} P(i) \log \frac{P(i)}{Q(i)}
  1073. ```
  1074. ```math
  1075. \text{JS}(P, Q) = \frac{1}{2} \text{KL}(P || M) + \frac{1}{2} \text{KL}(Q || M), M = \frac{P + Q}{2}
  1076. ```
  1077. Both functions are defined for non-negative numbers, and the logarithm is a key part of their computation.
  1078. ### Mixed Precision in Fused-Multiply-Add and Weighted Sums
  1079. The Fused-Multiply-Add (FMA) operation is a single operation that combines element-wise multiplication and addition with different scaling factors.
  1080. The Weighted Sum is its simplified variant without element-wise multiplication.
  1081. ```math
  1082. \text{FMA}_i(A, B, C, \alpha, \beta) = \alpha \cdot A_i \cdot B_i + \beta \cdot C_i
  1083. ```
  1084. ```math
  1085. \text{WSum}_i(A, B, \alpha, \beta) = \alpha \cdot A_i + \beta \cdot B_i
  1086. ```
  1087. In NumPy terms, the implementation may look like:
  1088. ```py
  1089. import numpy as np
  1090. def wsum(A: np.ndarray, B: np.ndarray, /, Alpha: float, Beta: float) -> np.ndarray:
  1091. assert A.dtype == B.dtype, "Input types must match and affect the output style"
  1092. return (Alpha * A + Beta * B).astype(A.dtype)
  1093. def fma(A: np.ndarray, B: np.ndarray, C: np.ndarray, /, Alpha: float, Beta: float) -> np.ndarray:
  1094. assert A.dtype == B.dtype and A.dtype == C.dtype, "Input types must match and affect the output style"
  1095. return (Alpha * A * B + Beta * C).astype(A.dtype)
  1096. ```
  1097. The tricky part is implementing those operations in mixed precision, where the scaling factors are of different precision than the input and output vectors.
  1098. SimSIMD uses double-precision floating-point scaling factors for any input and output precision, including `i8` and `u8` integers and `f16` and `bf16` floats.
  1099. Depending on the generation of the CPU, given native support for `f16` addition and multiplication, the `f16` temporaries are used for `i8` and `u8` multiplication, scaling, and addition.
  1100. For `bf16`, native support is generally limited to dot-products with subsequent partial accumulation, which is not enough for the FMA and WSum operations, so `f32` is used as a temporary.
  1101. ### Auto-Vectorization & Loop Unrolling
  1102. On the Intel Sapphire Rapids platform, SimSIMD was benchmarked against auto-vectorized code using GCC 12.
  1103. GCC handles single-precision `float` but might not be the best choice for `int8` and `_Float16` arrays, which have been part of the C language since 2011.
  1104. | Kind | GCC 12 `f32` | GCC 12 `f16` | SimSIMD `f16` | `f16` improvement |
  1105. | :------------------------ | -----------: | -----------: | ------------: | ----------------: |
  1106. | Inner Product | 3,810 K/s | 192 K/s | 5,990 K/s | __31 x__ |
  1107. | Cosine Distance | 3,280 K/s | 336 K/s | 6,880 K/s | __20 x__ |
  1108. | Euclidean Distance ² | 4,620 K/s | 147 K/s | 5,320 K/s | __36 x__ |
  1109. | Jensen-Shannon Divergence | 1,180 K/s | 18 K/s | 2,140 K/s | __118 x__ |
  1110. ### Dynamic Dispatch
  1111. Most popular software is precompiled and distributed with fairly conservative CPU optimizations, to ensure compatibility with older hardware.
  1112. Database Management platforms, like ClickHouse, and Web Browsers, like Google Chrome,need to run on billions of devices, and they can't afford to be picky about the CPU features.
  1113. For such users SimSIMD provides a dynamic dispatch mechanism, which selects the most advanced micro-kernel for the current CPU at runtime.
  1114. <table>
  1115. <tr>
  1116. <th>Subset</th>
  1117. <th>F</th>
  1118. <th>CD</th>
  1119. <th>ER</th>
  1120. <th>PF</th>
  1121. <th>4FMAPS</th>
  1122. <th>4VNNIW</th>
  1123. <th>VPOPCNTDQ</th>
  1124. <th>VL</th>
  1125. <th>DQ</th>
  1126. <th>BW</th>
  1127. <th>IFMA</th>
  1128. <th>VBMI</th>
  1129. <th>VNNI</th>
  1130. <th>BF16</th>
  1131. <th>VBMI2</th>
  1132. <th>BITALG</th>
  1133. <th>VPCLMULQDQ</th>
  1134. <th>GFNI</th>
  1135. <th>VAES</th>
  1136. <th>VP2INTERSECT</th>
  1137. <th>FP16</th>
  1138. </tr>
  1139. <tr>
  1140. <td><a href="https://en.wikipedia.org/wiki/Xeon_Phi#Knights_Landing">Knights Landing</a> (Xeon Phi x200, 2016)</td>
  1141. <td colspan="2" rowspan="9" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1142. <td colspan="2" rowspan="2" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1143. <td colspan="17" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1144. </tr>
  1145. <tr>
  1146. <td><a href="https://en.wikipedia.org/wiki/Xeon_Phi#Knights_Mill">Knights Mill</a> (Xeon Phi x205, 2017)</td>
  1147. <td colspan="3" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1148. <td colspan="14" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1149. </tr>
  1150. <tr>
  1151. <td>
  1152. <a href="https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#Skylake-SP_(14_nm)_Scalable_Performance">Skylake-SP</a>,
  1153. <a href="https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#Mainstream_desktop_processors">Skylake-X</a> (2017)
  1154. </td>
  1155. <td colspan="4" rowspan="11" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1156. <td rowspan="4" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1157. <td colspan="3" rowspan="4" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1158. <td colspan="11" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1159. </tr>
  1160. <tr>
  1161. <td><a href="https://en.wikipedia.org/wiki/Cannon_Lake_(microarchitecture)">Cannon Lake</a> (2018)</td>
  1162. <td colspan="2" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1163. <td colspan="9" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1164. </tr>
  1165. <tr>
  1166. <td><a href="https://en.wikipedia.org/wiki/Cascade_Lake_(microarchitecture)">Cascade Lake</a> (2019)</td>
  1167. <td colspan="2" rowspan="2" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1168. <td rowspan="2" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1169. <td colspan="8" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1170. </tr>
  1171. <tr>
  1172. <td><a href="https://en.wikipedia.org/wiki/Cooper_Lake_(microarchitecture)">Cooper Lake</a> (2020)</td>
  1173. <td style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1174. <td colspan="7" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1175. </tr>
  1176. <tr>
  1177. <td><a href="https://en.wikipedia.org/wiki/Ice_Lake_(microarchitecture)">Ice Lake</a> (2019)</td>
  1178. <td colspan="7" rowspan="3" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1179. <td rowspan="3" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1180. <td colspan="5" rowspan="3" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1181. <td colspan="2" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1182. </tr>
  1183. <tr>
  1184. <td><a href="https://en.wikipedia.org/wiki/Tiger_Lake_(microarchitecture)">Tiger Lake</a> (2020)</td>
  1185. <td style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1186. <td style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1187. </tr>
  1188. <tr>
  1189. <td><a href="https://en.wikipedia.org/wiki/Rocket_Lake">Rocket Lake</a> (2021)</td>
  1190. <td colspan="2" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1191. </tr>
  1192. <tr>
  1193. <td><a href="https://en.wikipedia.org/wiki/Alder_Lake">Alder Lake</a> (2021)</td>
  1194. <td colspan="2" style="background:#FFB;color:black;vertical-align:middle;text-align:center;">Partial</td>
  1195. <td colspan="15" style="background:#FFB;color:black;vertical-align:middle;text-align:center;">Partial</td>
  1196. </tr>
  1197. <tr>
  1198. <td><a href="https://en.wikipedia.org/wiki/Zen_4">Zen 4</a> (2022)</td>
  1199. <td colspan="2" rowspan="3" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1200. <td colspan="13" rowspan="3" style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1201. <td colspan="2" style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1202. </tr>
  1203. <tr>
  1204. <td><a href="https://en.wikipedia.org/wiki/Sapphire_Rapids_(microprocessor)">Sapphire Rapids</a> (2023)</td>
  1205. <td style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1206. <td style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1207. </tr>
  1208. <tr>
  1209. <td><a href="https://en.wikipedia.org/wiki/Zen_5">Zen 5</a> (2024)</td>
  1210. <td style="background:#9EFF9E;color:black;vertical-align:middle;text-align:center;">Yes</td>
  1211. <td style="background:#FFC7C7;color:black;vertical-align:middle;text-align:center;">No</td>
  1212. </tr>
  1213. </table>
  1214. You can compile SimSIMD on an old CPU, like Intel Haswell, and run it on a new one, like AMD Genoa, and it will automatically use the most advanced instructions available.
  1215. Reverse is also true, you can compile on a new CPU and run on an old one, and it will automatically fall back to the most basic instructions.
  1216. Moreover, the very first time you prove for CPU capabilities with `simsimd_capabilities()`, it initializes the dynamic dispatch mechanism, and all subsequent calls will be faster and won't face race conditions in multi-threaded environments.
  1217. ## Target Specific Backends
  1218. SimSIMD exposes all kernels for all backends, and you can select the most advanced one for the current CPU without relying on built-in dispatch mechanisms.
  1219. That's handy for testing and benchmarking, but also in case you want to dispatch a very specific kernel for a very specific CPU, bypassing SimSIMD assignment logic.
  1220. All of the function names follow the same pattern: `simsimd_{function}_{type}_{backend}`.
  1221. - The backend can be `serial`, `haswell`, `skylake`, `ice`, `genoa`, `sapphire`, `turin`, `neon`, or `sve`.
  1222. - The type can be `f64`, `f32`, `f16`, `bf16`, `f64c`, `f32c`, `f16c`, `bf16c`, `i8`, or `b8`.
  1223. - The function can be `dot`, `vdot`, `cos`, `l2sq`, `hamming`, `jaccard`, `kl`, `js`, or `intersect`.
  1224. To avoid hard-coding the backend, you can use the `simsimd_kernel_punned_t` to pun the function pointer and the `simsimd_capabilities` function to get the available backends at runtime.
  1225. To match all the function names, consider a RegEx:
  1226. ```regex
  1227. SIMSIMD_PUBLIC void simsimd_\w+_\w+_\w+\(
  1228. ```
  1229. On Linux, you can use the following command to list all unique functions:
  1230. ```sh
  1231. $ grep -oP 'SIMSIMD_PUBLIC void simsimd_\w+_\w+_\w+\(' include/simsimd/*.h | sort | uniq
  1232. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_haswell(
  1233. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_ice(
  1234. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_neon(
  1235. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_serial(
  1236. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_sve(
  1237. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_haswell(
  1238. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_ice(
  1239. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_neon(
  1240. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_serial(
  1241. > include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_sve(
  1242. ```
  1243. ## License
  1244. Feel free to use the project under Apache 2.0 or the Three-clause BSD license at your preference.