Athlete-turned-politician and committee president Seiko Hashimoto told BBC Sport: “I believe that the possibility of these Games going on is 100 per cent that we will do this.”
“The question right now is how are we going to have an even more safe and secure Games.
“The Japanese people are feeling very insecure and at the same time probably feel some frustration at us talking about the Olympics and I think that is giving rise to more voices opposing having the Games in Tokyo.
“The biggest challenge will be how we can control and manage the flow of people. If an outbreak should happen during the Games times that amounts to a crisis or an emergency situation then I believe we must be prepared to have these Games without any spectators.”
A majority of the Tokyo Metropolitan Assembly feel the same way, the Tokyo Shimbun newspaper reported on Thursday.
Prime minister Yoshihide Suga is likely to call a snap election after the Olympic and Paralympic Games, the Asahi newspaper reported, showing his resolve to push ahead with the event.
Foreign spectators have already been barred from the Games and officials are undecided if Japanese fans will be allowed into venues.
Economy minister Yasutoshi Nishimura pointed out on Thursday that excited fans, shouting and hugging could pose a risk.
Towns and cities set to host Olympic training or events have increasingly expressed concerns, amid worries visitors will spread variant strains of the virus and drain medical resources.
The government of Ota City has been inundated with complaints by residents over a decision to give preferential Covud vaccinations to city and hotel staff working with Australian athletes, it was reported.
The city, about 50 miles north-west of Tokyo, is the site of a training camp for Australia's softball team, which this week became the first national team to arrive in Japan.
Kurume City in the southern prefecture of Fukuoka pulled out of hosting Kenya's pre-Olympics training camp, the African nation's Olympics committee said on Wednesday.
A player on Ghana's Under-24 team tested positive for the coronavirus after arriving in Japan for a friendly match, the Japan Football Association said on Thursday.
Japan is battling a fourth wave of coronavirus eight weeks out from the scheduled start of the Games, but the country's vaccine rollout has been slow and 10 regions including Tokyo are under a state of emergency until June 20.
While Japan has avoided the large-scale infections suffered by many other nations, severe cases are rising in the latest outbreak. More than 746,000 cases have been recorded and more than 13,000 deaths.
Additional reporting by Reuters.