Muhammad Zahir, 25, told reporters on the dock at the Indonesian port of Merak that, as 150 refugees floated in the ocean, one boat came close enough for the crew to speak to them, only to say they would not pick them up.

The refugees, all Hazaras from Afghanistan and Pakistan, told traumatic stories of watching relatives die in the ocean after their flimsy vessel sank and the Indonesian crew swam away.

Mr Zahir said his brother and his sister, Hameeda, had both died, along with perhaps 10 other women and children.

“My sister, she was 29 years old. I was about to help her. But inside the water without any facilities, how could we help her? I was like, 'No, please don't go'.”

“People are gone, my brother, my sister gone, so how can we go back to Indonesia? Somebody help us please,” he said, crying.

A 10-year-old boy, Omed, sat on deck looking dazed. Other refugees said he had made the trip with his father, cousin and uncle, all of whom had died in the water. His mother had stayed in Afghanistan.

The refugees confirmed that the boat had been arranged by people smuggler Haji Ghulam, a Pakistani also known as Hassan, who is based in Indonesia. They said they had only ever heard his voice on the phone.

He had charged them $US5500 per person for safe passage to Australia.

“He told us, and lying and saying everything is clear, 'You guys aren't going to face a problem,” Mr Zahir said.

Another rescued asylum seeker, Reza, said he had worked as a translator and in providing security for international troops in Afghanistan, but that he and his family had been harassed by the Taliban, and they had been forced to flee the country.

“They threatened our families and had to move them to Pakistan and we had no security, we couldn't travel,” he said.

About 55 asylum seekers were rescued from the ocean, about 47 on the first Indonesian search and rescue boat, and the rest arrived on a second boat operated by Indonesian police.

On that boat is one refugee who died. It is not clear if this happened after he or she was pulled from the water.

After landing, the refugees were taken to the Feri Merak hotel in the town of Merak to be processed by the Indonesian immigration department. It is not clear what will happen next, but it is likely they will be sent to a detention centre.

“We are not happy to go,” Mr Zahir said. “We are ready to kill ourselves to jump in the water. We're not going in Indonesian immigration.”

He said to live in detention you needed money, and they had none because they had sold everything to pay the people smugglers to come to Australia.