In recent years taiwan government has made the best effort in adding science technology and creative idea into chinese traditional culture and arts to produce a new industrial and commercial product which is highly competitive on market, and can create considrable business opportunity over the world. taiwan puppet show (budaixi) [1]is a puppet controlled by hand. the stage for puppet performance is in the from the real stage but with reduced scale according to the size ratio of the real stage to human. the purpose of this research project is to virtualize the taiwan puppet show. the researchcan be divided into two phases. in the first phase some rules were established by induction method, and in the second phase the computer interaction technique was introduced to develop application software which possesses the nature of the entertainment and value of traditional culture. this paper aimed at the second phase of the research project in which the detail of the design idea employed and the practice are introduced. the outcome of the research not only benefited the preservation of chinese traditional culture, but also developed the '' finger control method '' for "augmented reality" which can be employed on other complicated interactive operation.