Screen LibraryStatistical significance testing for natural language processing /
Saved in:
| Main Authors: | |
|---|---|
| Group Author: | ; ; |
| Published: |
Morgan & Claypool Publishers,
|
| Publisher Address: | [San Rafael, California] : |
| Publication Dates: | [2020] |
| Literature type: | Book |
| Language: | English |
| Series: |
Synthesis lectures on human language technologies,
45 |
| Subjects: | |
| Carrier Form: | xvii, 98 pages : illustrations (some color) ; 24 cm. |
| Bibliography: | Includes bibliographical references (pages 81-96). |
| ISBN: |
9781681737959 1681737957 |
| Index Number: | QA76 |
| CLC: | H087 |
| Call Number: | H087/D786 |
| Contents: |
Intro -- Preface -- Acknowledgments -- Introduction -- Statistical Hypothesis Testing -- Hypothesis Testing -- P-Value in the World of NLP -- Statistical Significance Tests -- Preliminaries -- Parametric Tests -- Nonparametric Tests -- Statistical Significance in NLP -- NLP Tasks and Evaluation Measures -- Decision Tree for Significance Test Selection -- Matching Between Evaluation Measures and Statistical Significance Tests -- Significance with Large Test Samples -- Deep Significance -- Performance Variance in Deep Neural Network Models -- A Deep Neural Network Comparison Framework Existing Methods for Deep Neural Network Comparison -- Almost Stochastic Dominance -- Empirical Analysis -- Error Rate Analysis -- Summary -- Replicability Analysis -- The Multiplicity Problem -- A Multiple Hypothesis Testing Framework for Algorithm Comparison -- Replicability Analysis with Partial Conjunction Testing -- Replicability Analysis: Counting -- Replicability Analysis: Identification -- Synthetic Experiments -- Real-World Data Applications -- Applications and Data -- Statistical Significance Testing -- Results -- Results Summary and Overview -- Open Questions and Challenges |