The desire of looking for a more natural and efficient interactive method that human cantered are increasing recently. according to this status, this paper try to explore a new human-computer interactive system based on vision recognition, which more natural and more conforms to human behavior characteristic. this new system adopts RBF artificial neural network and hidden markov model, locates user's input focus combined with user head and the direction where the finger pointed to. in this way, the recognition accuracy and recognition speed of system is improved. it has wide appllication prospect in some field such as multimedia teaching, computer game, robot control and home electrical appliance control. first, capture human point action by camera and convert is to gray image. then identify human face and hand from image. input characcteristic values which extract from head image to the RBF neural network, gain head's level angle and vertical angle. at the same time, finger action be divided into three phase and build their markov model respectively in order to separate the hold-phase and get the direction that finger pointed to. by this two parameters, the human-computer interaction focus in camera's coordinate system be determined.