To construct color design techniques with respect to color memory, it is necessary to know the manner in which each color shifts in color memory. Previous studies report that the mean data of the color memory shift, but not all data can be utilized for color design, as there are high individual differences. In this study, we investigated what color data can be utilized in color design, and what data can be interpolated in color space. We performed a reproduction experiment of focal color and color memory as well as the categorical naming experiment. Results showed that the data of colors that can be recognized as a common color category can be used in color design. Results also showed that it might be possible to interpolate such data using data of neighborhood colors. For data of colors recognized as different color categories depending on observers, results could be categorized into three groups.