Improved Training Of Mixture-Of-Experts Language GANs
- Resource Type
- Conference
- Authors
- Chai, Yekun; Yin, Qiyue; Zhang, Junge
- Source
- ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2023 - 2023 IEEE International Conference on. :1-5 Jun, 2023
- Subject
- Bioengineering
Communication, Networking and Broadcast Technologies
Computing and Processing
Signal Processing and Analysis
Training
Image synthesis
Signal processing
Benchmark testing
Generative adversarial networks
Generators
Acoustics
Language GANs
Text Generation
- Language
- ISSN
- 2379-190X
Despite the dramatic success in image generation, Generative Adversarial Networks (GANs) still face great challenges in text generation. The difficulty in generator training arises from the limited representation capacity and uninformative learning signals obtained from the discriminator. In this work, we (1) first empirically show that the multi-generator approach is able to enhance the representation capacity of the generator for sequence GANs and (2) harness the Feature Statistics Alignment (FSA) paradigm to render fine-grained learning signals to advance the generator training. Specifically, FSA forces the mean statistics of the distribution of fake data to approach that of real samples as close as possible in the finite-dimensional feature space. Empirical study on synthetic and real benchmarks shows the superior performance in quantitative evaluation and demonstrates the effectiveness of our approach to adversarial text generation.