When choosing a heater, both the climate where you live and the amount of use you expect to get out of it are likely to make a difference to your choice. The demand for heating not only affects your long-term heating costs, but also how serious you need to be about keeping your entire house warm.
If you only need to heat a small area of your home for a week or two each year you should be able to get away with using just a small electric heater. They’re cheap to buy, don’t always need to be permanently installed and the cost to run them on an occasional basis is more or less negligible in the long term. It's much easier to use an electric heater over gas or solid fuel for occasional heating needs, because the vast majority of homes in Australia have ready access to power and you won't need to install any piping or completely permanent fixtures. If you start to look at connecting a gas line, your investment in a heater that will only be used for a handful of days each year will increase dramatically.
In some areas of Australia, like Melbourne, summers can get incredibly hot and winters can get incredibly cold. This plays a role in the type of heater you need, because while you’re not heating all year round, you will undoubtedly be using your heater for long periods of the day. For this reason a gas heater, while more expensive in the initial stages, may save you a fair bit in the longer term.
The other way to think about this is that if you live in an area that experiences hot summers and cold winters, a reverse cycle air conditioner is a convenient and relatively efficient way to install a single system that can both heat and cool your home.
If you need year-round heating for your home, you need to be serious about your choice and ensure that you're choosing a system that's both highly efficient and comfortable. Electric options are a fairly expensive choice for year-round heating, and wherever possible, central gas heating or hydronic central heating systems are a much wiser choice. These types of systems will give you great results, can be used for long periods and will cost a fraction of what electric options might cost.
Good passive design can largely do away with the need for excessive artificial heating - if you're starting from scratch, it's a very good idea to think about how best to minimise your reliance on artificial means of heating, and how to make the most of available sunlight.