The four countries fell short on corruption, political representation, economic management and respect for human rights, said the survey by the UN's Economic Commission for Africa.
Cameroon, Angola, Kenya and Nigeria are ranked as the most corrupt of the 28 countries surveyed.
The report's authors interviewed 50 000 households in the 28 countries surveyed in three phases between 2002 and early 2004. Two thousand experts were also interviewed to produce the first evaluation of how African governments are managed.
The Economic Commission for Africa chose the 28 countries out of Africa's 53 countries because the governments agreed to be surveyed.
Kingsley Amoako, executive director of the Economic Commission for Africa, said the survey showed African governments needed to be more democratic and build stronger institutions.
"These findings underpin the need for a capable, democratic state with strong institutions promoting the public interest," Amoako said.
Few Africans trust the police or think public services are efficient, the report said.
"Police and prison services violate the rights of citizens with impunity in several countries," the report said. "These agencies - especially the police - engage in torture, extra judicial killings and ill treatment of suspects awaiting trial."
It also said that in Kenya, Ethiopia, Chad, Zimbabwe and Malawi, "there are doubts about the commitment of government agencies to respect and implement the rule of law."
Titled, "Striving for Good Governance in Africa" the report said a third of the people surveyed say their parliaments are weak and fail to keep governments in check.
In Cameroon, almost half of those questioned said public services were very poor while starting a business in Mozambique can take close to half a year - 153 days - because of red tape.
Overall, the judiciary is seen as poor and in some countries like Burkina Faso it can take up to three or four years before a case is heard in court.
The survey looked at seven key areas, including political representation, corruption, human rights and economic management.
For more information visit the African Governance Report website at www.uneca.org. - Sapa-AP