en:infra-convert:user:terms
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | Next revision Both sides next revision | ||
|
en:infra-convert:user:terms [2019/10/02 14:13] me [Measurement (sizes-)] |
en:infra-convert:user:terms [2019/10/07 13:34] me |
||
|---|---|---|---|
| Line 31: | Line 31: | ||
| **M**\\ | **M**\\ | ||
| • [[en:infra-convert:user:terms#Measurement (sizes-)]]\\ | • [[en:infra-convert:user:terms#Measurement (sizes-)]]\\ | ||
| + | **N**\\ | ||
| + | • [[en:infra-convert:user:terms#Numerical precision]]\\ | ||
| **P**\\ | **P**\\ | ||
| • [[en:infra-convert:user:terms#Project]]\\ | • [[en:infra-convert:user:terms#Project]]\\ | ||
| Line 616: | Line 618: | ||
| | NIST-19: National Institute of Standards and Technology, Gaithersburg, Maryland, USA: NIST Handbook 44 : Specifications, Tolerances, and Other Technical Requirements for Weighing and Measuring Devices (Appendix C. General Tables of Units of Measurement) 2019.\\ *¹ Available from program version 1.2.0.23\\ *² Available from program version 1.3.9.5\\ *³ Available from program version 1.3.9.8 ||||| | | NIST-19: National Institute of Standards and Technology, Gaithersburg, Maryland, USA: NIST Handbook 44 : Specifications, Tolerances, and Other Technical Requirements for Weighing and Measuring Devices (Appendix C. General Tables of Units of Measurement) 2019.\\ *¹ Available from program version 1.2.0.23\\ *² Available from program version 1.3.9.5\\ *³ Available from program version 1.3.9.8 ||||| | ||
| + | \\ | ||
| + | |||
| + | ---- | ||
| + | |||
| + | \\ | ||
| + | |||
| + | ==== Numerical precision ==== | ||
| + | |||
| + | The numerical precision is relevant for the representation of numerical values. If a technical fact is described with numerical values, usually in the form of physical quantities or →[[en:infra-convert:user:terms#measurement_sizes|measurement sizes]], the | ||
| + | accuracy of the displayed number should be in the same order of magnitude as the inaccuracy of the measure itself. | ||
| + | |||
| + | For example, the maximum inaccuracy range of the value 23, if no further information is available, is 22.5 to <23.5. Finally, the value may have resulted from a rounding operation. The last significant digit, here the "3", is the digit that was rounded or may have been rounded. | ||
| + | |||
| + | |< 100% 15% 15% 30% 25% 15% >| | ||
| + | ^ Value ^ Number of relevant digits ^ Rounding range (absolute inaccuracy range) ^ Size of the absolute inaccuracy range ^ Size of relative inaccuracy ^ | ||
| + | | 2**3** | 2 | 22,5 to <23,5 | 1 | 4,35 % | | ||
| + | | 2,**3** | 2 | 2,25 to <2,35 | 0,1 | 4,35 % | | ||
| + | | 23,**0** | 3 | 22,95 to <23,05 | 0,1 | 0,435 % | | ||
| + | | 2**3**00 | 2 | 2250 to <2350 | 100 | 4,35 % | | ||
| + | | 0,00023**0** | 3 | 0,0002295 to <0,0002305 | 0,000001 | 0,435 % | | ||
| + | |||
| + | The number of significant or relevant digits in a decimal number can be identified as follows (see also "DIN EN ISO 80000-1:2013-08, Chapter 7.3.4 Deviation and uncertainty"): | ||
| + | |||
| + | - The number is viewed from left to right. The first non-zero digit is the first relevant digit. | ||
| + | - Following zeros are also relevant, especially if they are to the right of the decimal point. | ||
| + | - Zeros to the left of the comma are only relevant if at least one other digit, including zero, follows to the right of the comma. | ||
| + | The numerical accuracy common in technology ("slide rule accuracy") is achieved by rounding a value so that three significant digits remain. Unless the first significant digit is a one, four significant digits should remain. | ||
| \\ | \\ | ||
en/infra-convert/user/terms.txt · Last modified: 2023/10/20 16:02 by me
Page Tools
