{"id":653,"date":"2023-02-03T14:51:25","date_gmt":"2023-02-03T14:51:25","guid":{"rendered":"https:\/\/public.ksc.nasa.gov\/mns\/?page_id=653"},"modified":"2024-07-19T14:18:21","modified_gmt":"2024-07-19T18:18:21","slug":"faq","status":"publish","type":"page","link":"https:\/\/public.ksc.nasa.gov\/mns\/faq\/","title":{"rendered":"FAQ"},"content":{"rendered":"\n
<\/span>\"NASA
\n

FAQ<\/h1><\/div>\n<\/div><\/div>\n\n\n\n

<\/p>\n\n\n\n

Bolded<\/em><\/strong> words or phrases are key terms or acronyms defined under References or concepts discussed in NASA-STD-7009 or NASA-HDBK-7009.<\/em><\/p>\n\n\n

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is the purpose of this website (i.e., why this website exists)?<\/span><\/div>

To communicate, promote, and provide a medium for collaboration on the best practices in the development and use of models and simulations (M&S).<\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is basis for this content?<\/span><\/div>

The initial content for this website starts from the NASA Standard for Models and Simulations,\u00a0NASA-STD-7009<\/a>, which is broadly applicable to all types of M&S.<\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is span of M&S disciplines?<\/span><\/div>

The intent, for this information on the practice of modeling and simulation, is to be applicable to a broad range of modeling and simulation types and applications, not a single type or application.\u00a0 However, the practice of a single type or specific application of modeling, which might be broadened to a wider range of types or applications, is also valuable.<\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is M&S Planning & the M&S Life Cycle?<\/span><\/div>

Planning the development and use of an M&S is much like planning any other (hardware or software or system) project.\u00a0 Whether implicit or explicit, the development and subsequent use follows a set of ideally sequential phases with the potential for recursive loops to earlier phases, depending on many potential influences including the success of development and interim testing.

For more, see:\u00a0M&S Planning<\/a><\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is M&S Criticality?<\/span><\/div>

M&S Criticality is always with respect to the real-world decisions made from M&S results.\u00a0 A\u00a0critical decision<\/strong>\u00a0is defined as \u201cthe selection of a course-of-action related to design, development, manufacturing, ground, or flight operations that may significantly impact human safety or mission success, as measured by program\/project-defined criteria\u201d (NASA-STD-7009).

For more, see:\u00a0\u00a0M&S Criticality<\/a><\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is M&S Intended Use?<\/a><\/span><\/div>

The\u00a0intended use<\/strong>\u00a0of an M&S is \u201cthe expected purpose and application of an M&S\u201d and is best determined as early as possible in the M&S life cycle as a standard-bearer for development, even if it is modified at a later time.\u00a0 The intended use is also a concise communication mechanism of what the M&S does.

For more, see:\u00a0M&S Intended Use<\/a><\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is Reviewing & Testing?<\/span><\/div>

Reviewing and testing of an M&S can occur frequently throughout its development phases.\u00a0 From a rigorous perspective, a review of model development could occur at the end of each phase, to ensure all the processes and products of each phase are acceptably complete or ready to support the subsequent phase.\u00a0 A more common approach includes formal reviews at the completion of model design, called\u00a0conceptual (or design) validation<\/strong>, and at the completion of\u00a0verification<\/strong>\u00a0and\u00a0empirical validation<\/strong>.\u00a0\u00a0Conceptual validation<\/strong>\u00a0is typically a review between the modelers and the customers or stakeholders of the model, ideally prior to the start of model construction, to ensure the conceptual model or model design adequately represents the real world from the perspective of the intended uses of the M&S.

Model testing, after the completion of Model Construction, includes the independent tasks of verification and validation.\u00a0Verification<\/strong>\u00a0is the process of determining the extent to which an M&S is compliant with its requirements and specifications as detailed in its conceptual models, mathematical models, or other constructs.\u00a0Validation<\/strong>\u00a0is the process of determining the degree to which a model or a simulation is an accurate representation of the real world from the perspective of the\u00a0intended uses<\/strong>\u00a0of the M&S. Specifically accomplishing verification prior to empirical validation is, arguably, the most effective and efficient.\u00a0 Attempting empirical validation prior to full model verification is a highly risky practice, is to be avoided, and likely induces inefficiencies requiring repeated verification and validation cycles.
With the successful completion of verification and validation, the M&S is ready for official release and use.<\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is Official M&S Release?<\/span><\/div>

Releasing an M&S is more than simply making it available for use. At the successful completion of M&S testing (i.e., the distinctly independent tasks of verification and validation), the\u00a0permissible uses<\/strong>\u00a0of the M&S are documented, along with or as part of the\u00a0M&S User\u2019s Guide<\/strong>.

Additionally, a set of \u2018test suites\u2019 are now available (for future\u00a0regression tests<\/strong>) along with procedures for M&S use. Both the \u2018test suites\u2019 and \u2018M&S Use Procedures\u2019 are derived from those used in M&S verification and validation testing.<\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is M&S User\u2019s Guide?<\/span><\/div>

An M&S User\u2019s Guide is a document to help install, use, or troubleshoot an M&S.\u00a0 The\u00a0user\u2019s guide<\/strong>\u00a0typically include the M&S Statement of\u00a0Intended Use<\/strong>, a list of model\u00a0assumptions<\/strong>\u00a0and\u00a0abstractions<\/strong>, descriptions of how the model works, and the\u00a0permissible uses<\/strong>\u00a0of the model, including the verified and validated\u00a0limits of model operation (use)<\/strong>.
NASA-HDBK-7009, Appendix C, shows a content outline for an\u00a0M&S User\u2019s Guide<\/a>.<\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is Use Assessment?<\/span><\/div>

An M&S\u00a0Use Assessment<\/strong>\u00a0determines the acceptability of an M&S for a\u00a0Proposed Use<\/strong>\u00a0by comparison with the\u00a0Permissible Uses<\/strong>\u00a0of the M&S.

For more, see:\u00a0\u00a0M&S Use<\/a><\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is Reporting M&S Results?<\/span><\/div>

The NASA Standard for Models and Simulations (NASA-STD-7009) requires the\u00a0reporting<\/strong>\u00a0of supplementary information, along with the M&S results, to substantiate the\u00a0credibility<\/strong>\u00a0of the M&S results and understand the potential risks associated with accepting the results of the M&S use. The supplementary information includes:
\u25cf M&S\u00a0Criticality Assessment<\/strong>
\u25cf Analysis\u00a0Caveats<\/strong>
\u25cf M&S Results\u00a0Uncertainty<\/strong>
\u25cf M&S Results\u00a0Credibility<\/strong>\u00a0Assessment
\u25cf M&S Risk<\/strong>\u00a0Assessment
\u25cf M&S\u00a0Documentation<\/strong>

For more, see:\u00a0Reporting M&S Results<\/a><\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is M&S Credibility?<\/span><\/div>

The\u00a0credibility<\/strong>\u00a0of M&S-based results, i.e., \u201cthe quality to elicit belief or trust in M&S results,\u201d is not something that can be assessed directly. However, key factors of credibility may be assessed that inform a customer\u2019s (decision maker\u2019s) credibility assessment. These eight key factors are:
\u25cf Data Pedigree<\/strong>
\u25cf Verification<\/strong>
\u25cf Validation<\/strong>
\u25cf Input Pedigree<\/strong>
\u25cf Uncertainty Characterization<\/strong>
\u25cf Results Robustness<\/strong>
\u25cf M&S History<\/strong>
\u25cf M&S Process \/ Product Management.<\/strong>

For more, see:\u00a0M&S Results Credibility<\/a>\u00a0<\/p><\/div><\/div>

\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<\/path><\/svg>\n\t\t\t\t\t\t\t<\/span>\n\t\t\tWhat is M&S Risk?<\/span><\/div>

From an M&S perspective, a risk exists in the potential shortfalls in the M&S with respect to sufficiently representing the\u00a0RWS<\/strong>. In other words, if an M&S does not sufficiently represent the real world system for which it is providing results, then some amount of risk is incurred when using the M&S results to make real world decisions. An assessment of M&S risk is, therefore, warranted.

For more, see:\u00a0M&S Risk<\/a>\u00a0
The sources of M&S risk may originate in any phase of the\u00a0M&S Life Cycle<\/a>
Refer to the\u00a0NASA-HDBK-7009<\/a>:
\u25cf Section 5.6.3.2.2 M&S Risk Assessment.
\u25cf Appendix E \u2013 M&S Risk Assessment.<\/p><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"

Bolded words or phrases are key terms or acronyms defined under References or concepts discussed in NASA-STD-7009 or NASA-HDBK-7009.<\/p>\n","protected":false},"author":41,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_uag_custom_page_level_css":"","_lmt_disableupdate":"","_lmt_disable":"","advgb_blocks_editor_width":"","advgb_blocks_columns_visual_guide":"","site-sidebar-layout":"no-sidebar","site-content-layout":"page-builder","ast-site-content-layout":"full-width-container","site-content-style":"unboxed","site-sidebar-style":"unboxed","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"disabled","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"set","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"class_list":["post-653","page","type-page","status-publish","hentry"],"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","pp_force_visibility":null,"pp_subpost_visibility":null,"pp_inherited_force_visibility":null,"pp_inherited_subpost_visibility":null,"pp_post_mime_type":"","uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false,"gform-image-choice-sm":false,"gform-image-choice-md":false,"gform-image-choice-lg":false},"uagb_author_info":{"display_name":"Teresa Parrish","author_link":"https:\/\/public.ksc.nasa.gov\/mns\/author\/tparrish\/"},"uagb_comment_info":0,"uagb_excerpt":"Bolded words or phrases are key terms or acronyms defined under References or concepts discussed in NASA-STD-7009 or NASA-HDBK-7009.","coauthors":[],"author_meta":{"author_link":"https:\/\/public.ksc.nasa.gov\/mns\/author\/tparrish\/","display_name":"Teresa Parrish"},"relative_dates":{"created":"Posted 4 years ago","modified":"Updated 2 years ago"},"absolute_dates":{"created":"Posted on February 3, 2023","modified":"Updated on July 19, 2024"},"absolute_dates_time":{"created":"Posted on February 3, 2023 2:51 pm","modified":"Updated on July 19, 2024 2:18 pm"},"featured_img_caption":"","featured_img":false,"series_order":"","_links":{"self":[{"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/pages\/653","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/users\/41"}],"replies":[{"embeddable":true,"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/comments?post=653"}],"version-history":[{"count":14,"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/pages\/653\/revisions"}],"predecessor-version":[{"id":996,"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/pages\/653\/revisions\/996"}],"wp:attachment":[{"href":"https:\/\/public.ksc.nasa.gov\/mns\/wp-json\/wp\/v2\/media?parent=653"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}