{"id":651,"date":"2023-02-03T14:49:24","date_gmt":"2023-02-03T14:49:24","guid":{"rendered":"https:\/\/public.ksc.nasa.gov\/mns\/?page_id=651"},"modified":"2025-02-20T18:32:27","modified_gmt":"2025-02-20T23:32:27","slug":"ms-use","status":"publish","type":"page","link":"https:\/\/public.ksc.nasa.gov\/mns\/ms-use\/","title":{"rendered":"M&#038;S Use"},"content":{"rendered":"\n<div class=\"wp-block-cover alignfull\" style=\"min-height:220px;aspect-ratio:unset;\"><span aria-hidden=\"true\" class=\"wp-block-cover__background has-ast-global-color-8-background-color has-background-dim\"><\/span><img loading=\"lazy\" decoding=\"async\" width=\"1920\" height=\"522\" class=\"wp-block-cover__image-background wp-image-102\" alt=\"NASA models and simulations (M&amp;S)\" src=\"https:\/\/public.ksc.nasa.gov\/mns\/wp-content\/uploads\/sites\/20\/2022\/12\/HeroBG-1.jpg\" style=\"object-position:49% 75%\" data-object-fit=\"cover\" data-object-position=\"49% 75%\"\/><div class=\"wp-block-cover__inner-container is-layout-flow wp-block-cover-is-layout-flow\">\n<div class=\"wp-block-uagb-advanced-heading uagb-block-735c058e\"><h1 class=\"uagb-heading-text\">M&amp;S Use<\/h1><\/div>\n<\/div><\/div>\n\n\n\n<p><\/p>\n\n\n\n<p><em><strong>Bolded<\/strong>&nbsp;words or phrases are key terms or acronyms defined under References or concepts discussed in NASA-STD-7009 or NASA-HDBK-7009.<\/em><\/p>\n\n\n\n<p>The M&amp;S Use Phase (Phase E) is shown here with added detail in the context of the entire&nbsp;<a href=\"https:\/\/public.ksc.nasa.gov\/mns\/ms-lifecycle\/\" data-type=\"page\" data-id=\"638\">M&amp;S Life Cycle<\/a><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/public.ksc.nasa.gov\/-\/media\/MnS\/Use1.ashx?h=419&amp;w=1000&amp;la=en&amp;hash=DCC5E437D079D8CCC63B325E650CAF22\" alt=\"M&amp;S Use Phase\"\/><\/figure>\n\n\n\n<p>M&amp;S Use, taken as a broad and general phase in the&nbsp;<a href=\"https:\/\/public.ksc.nasa.gov\/mns\/ms-lifecycle\/\" data-type=\"page\" data-id=\"638\">M&amp;S Life Cycle<\/a>, is more than simply using the M&amp;S. There are some tasks to accomplish before and after the actual use, and so this phase is discussed in three sub-phases.<strong>Pre-Use<\/strong><br>The needs for overall M&amp;S Use are:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The Released M&amp;S<\/li>\n\n\n\n<li>The M&amp;S&nbsp;<strong>User\u2019s Guide<\/strong><\/li>\n\n\n\n<li>Procedures for M&amp;S Use<\/li>\n\n\n\n<li>The Infrastructure for M&amp;S Use (M&amp;S H\/W &amp; S\/W)<\/li>\n<\/ul>\n\n\n\n<p>Once these items are in hand, each proposed use of the M&amp;S must be evaluated against the&nbsp;<strong>permissible uses<\/strong>&nbsp;of the M&amp;S, as determined in M&amp;S development and testing. The following table depicts these similar elements for comparison in a use assessment.<\/p>\n\n\n\n<div class=\"wp-block-uagb-container uagb-block-76b42dff alignfull uagb-is-root-container\"><div class=\"uagb-container-inner-blocks-wrap\">\n<div class=\"wp-block-uagb-container uagb-block-eff23a4c\">\n<table class=\"wp-block-advgb-table advgb-table-frontend\"><tbody><tr><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);text-align:center\"><strong>Permissible Use(s) of Model<\/strong>&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Type of Use Intended.<br>\u2022 Implies the Type of Model.<br>\u2022 The application domain (discipline, area of study) of the Model.<br>\u2022 The Purpose of the Model.<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Model\u2019s&nbsp;<strong>Abstractions<\/strong>&nbsp;and&nbsp;<strong>Assumptions<\/strong>.<br>\u2022 Inclusions in the M&amp;S.<br>\u2022 Exclusions from the M&amp;S.<br>\u2022 Assumptions of M&amp;S form, fit, or function.<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Limits of Model Parameters, per<br>\u2022 Model design (including any computer H\/W or S\/W limitations).<br>\u2022 <strong>Verification.<\/strong><br>\u2022 <strong>Validation.<\/strong><\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Types of Outputs (Results) Produced, including:<br>\u2022 Accuracy.<br>\u2022 Precision.<br>\u2022 Uncertainty.<\/td><\/tr><\/tbody><\/table>\n<\/div>\n\n\n\n<div class=\"wp-block-uagb-container uagb-block-829da018\">\n<table class=\"wp-block-advgb-table advgb-table-frontend\"><tbody><tr><td style=\"color:var(--ast-global-color-5);background-color:var(--ast-global-color-0);text-align:center\">&nbsp;<strong>Proposed Use(s) of Model<\/strong><\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Type of Use Needed.<br>\u2022 Implied by the type of&nbsp;<strong>RWS<\/strong>.<br>\u2022 The application area (discipline, area of study) of the subject RWS.<br>\u2022 The purpose of proposed model use with respect to the RWS.<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Inclusions &amp; Fidelity Needed.<br>\u2022 Specific expectations of what is in, or expected of, the M&amp;S.<br>\u2022 The desired level of accuracy, precision, &amp;&nbsp;<strong>uncertainty<\/strong>&nbsp;of the M&amp;S.<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Desired Domain of Use.<br>\u2022 With respect to the RWS.<br>\u2022 Parameter values the model is expected to represent.<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Type of Results Needed, including:<br>\u2022 Accuracy.<br>\u2022 Precision.<br>\u2022 Uncertainty of Results.<\/td><\/tr><\/tbody><\/table>\n<\/div>\n<\/div><\/div>\n\n\n\n<p>It is also prudent to assess the&nbsp;<strong>criticality<\/strong>&nbsp;of the situation upon which the M&amp;S is used. Critical situations imply a greater need for rigor, control, and documentation (evidence).<\/p>\n\n\n\n<p><em>Note: this is the 2nd time Criticality is mentioned. The first time was at the initiation of model development, to help guide development. While a model may be developed for an initial intended use (critical or not), the use of the model may different to some degree. Therefore, this criticality assessment is for the \u2018use at hand.\u2019<\/em><\/p>\n\n\n\n<p>The last task prior to actual use of application of the M&amp;S is to determine or define the scenarios and explicit input for model execution. This can take a wide variety of data analysis methods, including design of experiments and stochastic variable modeling.<strong>Use<\/strong><\/p>\n\n\n\n<p>Strictly speaking, the actual \u2018use\u2019 of an M&amp;S only consists of setting&nbsp;up the M&amp;S for use and using (or running \/ executing) it.<strong>Post-Use<\/strong><\/p>\n\n\n\n<p>After running the M&amp;S, the resulting data is analyzed and then assessed and prepared for dissemination in reports or briefings. During data analysis, value is added to the results by also presenting an understanding of the results&nbsp;<strong>uncertainty<\/strong>&nbsp;and&nbsp;<strong>sensitivity<\/strong>.<\/p>\n\n\n\n<p>The NASA Standard for Models and Simulations also requires the&nbsp;<strong>reporting<\/strong>&nbsp;of supplementary information, along with the M&amp;S results, to substantiate the&nbsp;<strong>credibility<\/strong>&nbsp;of the M&amp;S results and understand the&nbsp;<strong>risks<\/strong>&nbsp;associated with accepting the results of the M&amp;S use.<\/p>\n\n\n\n<h3 class=\"wp-block-heading alignwide has-ast-global-color-0-color has-text-color\" id=\"reporting-results\"><strong>Reporting M&amp;S Results<\/strong><\/h3>\n\n\n\n<p>There is much more to reporting M&amp;S results than the answers output from its use. The risks potentially incurred during the development and use of the M&amp;S, including the factors influencing the credibility of those results, have implications to the acceptance of those results. Clearly conveying these risk elements and credibility factors, along with the criticality associated with the represented real world system, supports a robust decision-making process. Each of these aspects of M&amp;S results reporting is shown in the following diagram. Select each aspect in the diagram for additional details or refer to the appropriate sections of&nbsp;<a rel=\"noreferrer noopener\" href=\"https:\/\/standards.nasa.gov\/standard-models-and-simulations\" target=\"_blank\">NASA-STD-7009<\/a>&nbsp;or&nbsp;<a rel=\"noreferrer noopener\" href=\"https:\/\/standards.nasa.gov\/standard\/nasa\/nasa-hdbk-7009\" target=\"_blank\">NASA-HDBK-7009<\/a>. The [M&amp;S x] notations indicate requirement numbers in the NASA Standard.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/public.ksc.nasa.gov\/-\/media\/MnS\/Use2.ashx?h=701&amp;w=1000&amp;la=en&amp;hash=D80DFA5A3A752A8472C537FD40A1D5D0\" alt=\"M&amp;S Criticality \"\/><\/figure>\n\n\n\n<p><a href=\"https:\/\/public.ksc.nasa.gov\/mns\/ms-planning\/#M&amp;S-Criticality\" data-type=\"URL\" data-id=\"https:\/\/public.ksc.nasa.gov\/mns\/ms-planning\/#M&amp;S-Criticality\">M&amp;S Criticality<\/a>&nbsp;is best assessed early in both M&amp;S Development and M&amp;S Use to help inform the needed rigor in M&amp;S processes and products.<\/p>\n\n\n\n<h3 class=\"wp-block-heading alignwide has-ast-global-color-0-color has-text-color\"><strong>Analysis Caveats<\/strong><\/h3>\n\n\n\n<p><strong>Caveats<\/strong>&nbsp;are information included with M&amp;S results to provide cautions or prevent misinterpretation when evaluating, interpreting, or making decisions. This type of information includes:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Unachieved acceptance criteria<\/li>\n\n\n\n<li>Violation of assumptions<\/li>\n\n\n\n<li>Violation of the Limits of Operation<\/li>\n\n\n\n<li>Execution Warning and Error Messages<\/li>\n\n\n\n<li>Unfavorable outcomes from the proposed use assessments<\/li>\n\n\n\n<li>Unfavorable outcomes from Setup\/Execution Assessments<\/li>\n\n\n\n<li>Waivers to Requirements<\/li>\n<\/ol>\n\n\n\n<p>A placard to the M&amp;S results is warranted if the caveat is significant.<\/p>\n\n\n\n<p>The addition of a placard helps communicate the potential issue associated with the M&amp;S results.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/public.ksc.nasa.gov\/-\/media\/MnS\/UseCaution.ashx?h=237&amp;w=316&amp;la=en&amp;hash=F420B6AD578E27375BF35860AB0F29B5\" alt=\"Caution\"\/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading alignwide has-ast-global-color-0-color has-text-color\"><strong>M&amp;S Results Uncertainty<\/strong><\/h3>\n\n\n\n<p>M&amp;S\u2019s are neither perfect nor all-inclusive representations and so will have some&nbsp;<strong>uncertainty<\/strong>&nbsp;in the results from their use. Uncertainties describe the imperfect state of knowledge or a variability resulting from a variety of factors, including but not limited to lack of knowledge, applicability of information, physical variation, randomness or stochastic behavior, indeterminacy, judgment, and approximation. Uncertainties take many forms, stem from many sources, and should be addressed when reporting M&amp;S results. Assessing M&amp;S uncertainties starts with understanding the following:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Sources<\/li>\n\n\n\n<li>Locations<\/li>\n\n\n\n<li>Characteristics (Type)<\/li>\n\n\n\n<li>Magnitude<\/li>\n\n\n\n<li>How well known<\/li>\n\n\n\n<li>How they are used (incorporated) in the M&amp;S<\/li>\n\n\n\n<li>How they propagate through the M&amp;S<\/li>\n<\/ul>\n\n\n\n<p>For clarity, all M&amp;S results are to be accompanied with one of the following:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>A quantitative estimate of the uncertainty,<\/li>\n\n\n\n<li>A qualitative description of the uncertainty,<\/li>\n\n\n\n<li>A clear statement that no quantitative estimate or qualitative description of uncertainty is available.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading alignwide has-ast-global-color-0-color has-text-color\" id=\"ResultsCredibility\"><strong>M&amp;S Credibility<\/strong><\/h3>\n\n\n\n<p>M&amp;S Credibility is the quality to elicit belief or trust in M&amp;S results and cannot be determined directly; however, key factors exist that contribute to a decision-maker\u2019s credibility assessment. The eight factors in NASA\u2019s assessment of M&amp;S Results Credibility were selected from a long list of factors that potentially contribute to the credibility of M&amp;S results because<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>they were individually judged to be the key factors in this list<\/li>\n\n\n\n<li>they are nearly orthogonal, i.e., largely independent, factors<\/li>\n\n\n\n<li>they can be assessed objectively.<\/li>\n<\/ol>\n\n\n\n<p>These factors are organized in three groups, as follows:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>M&amp;S Development\n<ol class=\"wp-block-list\">\n<li><em>Data Pedigree: Is the pedigree (and quality) of the data used to develop the model adequate or acceptable?<\/em><\/li>\n\n\n\n<li><em>Verification: Were the models implemented correctly, per their requirements\/specifications?<\/em><\/li>\n\n\n\n<li><em>Validation: Did the M&amp;S results compare favorably to the referent data, and how close is the referent to the RWS?<\/em><\/li>\n<\/ol>\n<\/li>\n\n\n\n<li>M&amp;S Operations<ol><li><em>Input Pedigree: Is the pedigree (and quality) of the data used to setup and run the model adequate or acceptable?<\/em><\/li><li><em>Uncertainty Characterization: Is the uncertainty in the current M&amp;S results appropriately characterized? What are the sources of uncertainty in the results and how are they propagated through to the results of the analysis?<\/em><\/li><li><em>Results Robustness: How thoroughly are the sensitivities of the current M&amp;S results known?<\/em><\/li><\/ol>\n<ol class=\"wp-block-list\">\n<li>Supporting Evidence\n<ol class=\"wp-block-list\">\n<li><em>M&amp;S History: How similar is the current version of the M&amp;S to previous versions, and how similar is the current use of the M&amp;S to previous successful uses?<\/em><\/li>\n\n\n\n<li><em>M&amp;S Management: How well managed were the M&amp;S processes and products?<\/em><\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n\n\n\n<p>The following schema was developed to assess each of the credibility factors:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image alignwide size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"617\" src=\"https:\/\/public.ksc.nasa.gov\/mns\/wp-content\/uploads\/sites\/20\/2023\/04\/MSUseTable-1024x617.png\" alt=\"M&amp;S Use Table\" class=\"wp-image-912\" srcset=\"https:\/\/public.ksc.nasa.gov\/mns\/wp-content\/uploads\/sites\/20\/2023\/04\/MSUseTable-1024x617.png 1024w, https:\/\/public.ksc.nasa.gov\/mns\/wp-content\/uploads\/sites\/20\/2023\/04\/MSUseTable-300x181.png 300w, https:\/\/public.ksc.nasa.gov\/mns\/wp-content\/uploads\/sites\/20\/2023\/04\/MSUseTable-768x463.png 768w, https:\/\/public.ksc.nasa.gov\/mns\/wp-content\/uploads\/sites\/20\/2023\/04\/MSUseTable-1536x926.png 1536w, https:\/\/public.ksc.nasa.gov\/mns\/wp-content\/uploads\/sites\/20\/2023\/04\/MSUseTable.png 1764w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>The results of the credibility assessment may be presented to decision makers in a variety of forms. The following graphical methods are just two possibilities.<\/p>\n\n\n\n<p class=\"has-text-align-center\"><strong>Credibility Assessment \u2013 Bar Chart<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/public.ksc.nasa.gov\/-\/media\/MnS\/CredAssesment1.ashx?h=346&amp;w=522&amp;la=en&amp;hash=27C280C46F1164044F05ED38D63AA634\" alt=\"Credibility Assessment \u2013 Bar Chart\"\/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p class=\"has-text-align-center\"><strong>Credibility Assessment \u2013 Spider Plot (Radar Chart)<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/public.ksc.nasa.gov\/-\/media\/MnS\/SpiderPlot1.ashx?h=317&amp;w=522&amp;la=en&amp;hash=01420ACC7429E092625A93F0424BCC6C\" alt=\"Spider Plot (Radar Chart)\"\/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading alignwide has-ast-global-color-0-color has-text-color\"><strong>M&amp;S Documentation<\/strong><\/h3>\n\n\n\n<p>Documentation is one of the cornerstones of the&nbsp;<a rel=\"noreferrer noopener\" href=\"http:\/\/standards.nasa.gov\/standard-models-and-simulations\" target=\"_blank\">NASA Standard for Models and Simulations<\/a>, with the underlying theme that evidence is required to claim an activity is accomplished or a requirement is satisfied (i.e., if it is not documented, it is not done).<\/p>\n\n\n\n<p>The NASA Handbook for Models and Simulations Table 13 provides a method to synopsize the documentation (evidence) requirements of the Standard. For each of the \u2018documentation\u2019 requirements, the following information is requested:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Do any of the documents exist?<\/li>\n\n\n\n<li>If yes, what are the documents?<\/li>\n\n\n\n<li>What is the rationale for proceeding with the reported information?<\/li>\n<\/ul>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading alignwide has-ast-global-color-0-color has-text-color\" id=\"tech\"><strong>M&amp;S Technical Review<\/strong><\/h3>\n\n\n\n<p>Technical reviews of M&amp;S development and use processes and products help bolster the acceptability of M&amp;S-based analyses. Knowing the results (or findings) from existing or past technical reviews is, therefore, valuable.<\/p>\n\n\n\n<p>Technical reviews of M&amp;S development and use processes and products are valuable in determining the acceptability of M&amp;S-based analyses. Understanding the information in the following table provides supporting material in ascertaining the risks of accepting the results from an M&amp;S-based analysis.<\/p>\n\n\n\n<table class=\"wp-block-advgb-table alignwide advgb-table-frontend\"><tbody><tr><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">7009 Req&#8217;t       <\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Reporting Req&#8217;t                                   <\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Do any of the following exist? (Yes \/ No)&nbsp;<\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">If yes, what are they?<\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Rationale for proceeding with the reported information<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">[M&amp;S 36]<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\"><strong>Technical Review<\/strong><\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Review<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&#8211; What was reviewed?<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;&#8211; Depth of Review.<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;&#8211; Formality of Review.<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&#8211; Currency of Review.<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Reviewers<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&#8211; Expertise.<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&#8211; Independence.<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><\/tbody><\/table>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading alignwide has-ast-global-color-0-color has-text-color\" id=\"people\"><strong>People Qualifications<\/strong><\/h3>\n\n\n\n<p>The qualifications and experience of the people developing, testing, using, and analyzing M&amp;S can also provide supporting material in ascertaining the risks of accepting the results from an M&amp;S-based analysis.<\/p>\n\n\n\n<table class=\"wp-block-advgb-table alignwide advgb-table-frontend\"><tbody><tr><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">7009 Req&#8217;t    <\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Reporting Req&#8217;t<\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Do any of the following exist? (Yes \/ No)<\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">If yes, what are they?<\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Rationale for proceeding with the reported information&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">[M&amp;S 37]<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\"><strong>People Qualifications<\/strong><\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Developers.<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Testers.<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Users (Operators).<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Analysts.<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">&nbsp;<\/td><\/tr><\/tbody><\/table>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading alignwide has-ast-global-color-0-color has-text-color\" id=\"risk\"><strong>M&amp;S Risk<\/strong><\/h3>\n\n\n\n<p>The definition of&nbsp;<strong>M&amp;S Risk<\/strong>&nbsp;is \u201cthe potential for shortfalls with respect to sufficiently representing a&nbsp;<strong>Real World System<\/strong>.\u201d Risk may be induced anywhere in the M&amp;S Life Cycle, from creation to use. All of the additional attributes associated with the reporting of M&amp;S results may bolster or detract from the overall risk associated with accepting the M&amp;S-based results. Besides the aforementioned attributes, the existence and quality of&nbsp;<a href=\"#tech\" data-type=\"internal\" data-id=\"#tech\">M&amp;S technical reviews<\/a>&nbsp;and the&nbsp;<a href=\"#people\" data-type=\"internal\" data-id=\"#people\">qualifications of <\/a><a href=\"https:\/\/public.ksc.nasa.gov\/MnS\/Use#people\">the people<\/a>&nbsp;involved in M&amp;S development, use, and analysis.<em>NASA M&amp;S Use References:<\/em><\/p>\n\n\n\n<table class=\"wp-block-advgb-table alignwide advgb-table-frontend is-style-stripes\"><tbody><tr><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Topic<\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">NASA-STD-7009<\/td><td style=\"background-color:var(--ast-global-color-0);color:var(--ast-global-color-5);border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">NASA-HDBK-7009<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Use<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 4.3<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Sections 5.6<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Use Assessment<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\"><\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">5.6.1.2<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Criticality<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Appendix D<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\"><\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Caveats<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 4.3.8<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 5.6.3, Table 8<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Results Uncertainty<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Sections 4.2.7, 4.2.8, 4.3.3, 4.3.4<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 5.6.3.1.1 Appendix D.7<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Results Credibility Assessment<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Appendix E<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 5.6.3.2.1 Appendix D<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Risk Assessment Elements<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\"><\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 5.6.3.2.2<br>Table 14<br>Section 6.4, Table 20<br>Appendix E, Table 37<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Technical Review<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\"><\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 5.6.3.2 Table 11<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S People Qualifications<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\"><\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 5.6.3.2 Table 12<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Documentation<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\"><\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 5.6.3.2 Table 13<\/td><\/tr><tr><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">M&amp;S Results Reporting<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 4.3.8<\/td><td style=\"border-top-color:#000;border-right-color:#000;border-bottom-color:#000;border-left-color:#000\">Section 5.6.3.2.3<\/td><\/tr><\/tbody><\/table>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Bolded&nbsp;words or phrases are key terms or acronyms defined under References or concepts discussed in NASA-STD-7009 or NASA-HDBK-7009. The M&amp;S Use Phase (Phase E) is shown here with added detail in the context of the entire&nbsp;M&amp;S Life Cycle M&amp;S Use, taken as a broad and general phase in the&nbsp;M&amp;S Life Cycle, is more than simply [&hellip;]<\/p>\n","protected":false},"author":41,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_uag_custom_page_level_css":"","_lmt_disableupdate":"","_lmt_disable":"","advgb_blocks_editor_width":"","advgb_blocks_columns_visual_guide":"","site-sidebar-layout":"no-sidebar","site-content-layout":"page-builder","ast-site-content-layout":"full-width-container","site-content-style":"unboxed","site-sidebar-style":"unboxed","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"disabled","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"set","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"class_list":["post-651","page","type-page","status-publish","hentry"],"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","pp_force_visibility":null,"pp_subpost_visibility":null,"pp_inherited_force_visibility":null,"pp_inherited_subpost_visibility":null,"pp_post_mime_type":"","uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false,"gform-image-choice-sm":false,"gform-image-choice-md":false,"gform-image-choice-lg":false},"uagb_author_info":{"display_name":"Teresa Parrish","author_link":"https:\/\/public.ksc.nasa.gov\/mns\/author\/tparrish\/"},"uagb_comment_info":0,"uagb_excerpt":"Bolded&nbsp;words or phrases are key terms or acronyms defined under References or concepts discussed in NASA-STD-7009 or NASA-HDBK-7009. The M&amp;S Use Phase (Phase E) is shown here with added detail in the context of the entire&nbsp;M&amp;S Life Cycle M&amp;S Use, taken as a broad and general phase in the&nbsp;M&amp;S Life Cycle, is more than simply&hellip;","coauthors":[],"author_meta":{"author_link":"https:\/\/public.ksc.nasa.gov\/mns\/author\/tparrish\/","display_name":"Teresa Parrish"},"relative_dates":{"created":"Posted 4 years ago","modified":"Updated 2 years ago"},"absolute_dates":{"created":"Posted on February 3, 2023","modified":"Updated on February 20, 2025"},"absolute_dates_time":{"created":"Posted on February 3, 2023 2:49 pm","modified":"Updated on February 20, 2025 6:32 pm"},"featured_img_caption":"","featured_img":false,"series_order":"","_links":{"self":[{"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/pages\/651","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/users\/41"}],"replies":[{"embeddable":true,"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/comments?post=651"}],"version-history":[{"count":40,"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/pages\/651\/revisions"}],"predecessor-version":[{"id":997,"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/pages\/651\/revisions\/997"}],"wp:attachment":[{"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/media?parent=651"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}