How should we define political institutions?

According to the Stanford Encyclopedia, social institutions are commonly understood as sets of rules and norms that organise human activities within a society. Following that definition, political institutions could be defined as sets of rules and norms that organise political activity.
Institutions do not necessarily have to be written down, but they have to be enforced or commonly accepted (or both). In a narrower meaning, “political institutions” also describe the organisations that shape the actions within a political system, such as parties, courts, trade unions, etc.

Is there a point in analysing political systems by the formal rules that apply, unless these rules actually mean something to the people?

If any level of political analysis exists that can draw conclusions from unaccepted formal rules, it could make sense to study those rules.  In reverse: Above statement “There is no point …” only holds true, when no such level of analysis exists. This is not the case.

First, the degree to which formal rules are valued might be used to evaluate the degree of legitimation of a political system. Rules and laws should be the result of a societal consent.
If a large quantity of citizens does not agree with some of the laws, these laws are either outdated (and have been failed to be revised) or have not been the result of a consent in the first place. In either case one might say that the rules are illegitimate. Good examples are the restrictions of freedom of speech and freedom of assembly in Egypt, Tunisian, and Syria that increased dissatisfaction with the political systems and eventually led to the disestablishment of the Tunisian and Egyptian government.

Second, the acceptance of formal rules might also say something about the effectiveness of a political system. For instance, the fact that up to 50% of citizens in some Latin American countries ignore their tax liabilities might be a good indicator for the overall effectiveness of the systems. In consequence this also explains fiscal difficulties in these countries.

Third, the existence of unaccepted formal rules might also point out to issues that should be discussed and changed by the political actors. Examples are the copyright laws for digital contents that are generally ignored by significant parts of the populations in many states. This might suggest that law makers in those countries have not yet managed to cope with new information systems.
Another example is the prohibition of online gambling in Germany. This law is routinely ignored by millions of Germans who play poker or do sports betting online without any fear of persecution. The main consequence of the law is that all gambling platforms have their seat in foreign countries, where they are out of reach of the German judicial system. Eventually, ban of online gambling therefore even decreases the possibilities of German political actors to manage the problem.

In conclusion it should have become apparent that there is a point in analysing political systems also by the rules that do not mean anything to the people.

What potential advantages are there in studying political institutions from a cultural perspective?

Political culture is commonly understood as the view of a nation’s citizens towards politics. It affects their perceptions of political legitimacy. Ideally, the political culture underpins the legitimacy of a political system, thereby encouraging people to accept the enforcement of law and shaping the political institutions to most precisely match their specific needs.

When political culture is understood as the view of a nation’s citizens towards politics, this implies that studying institutions from a cultural perspective means to adapt the perspective of the citizens. This might be very helpful to understand why some institutions are more accepted than others. From this understanding one could draw conclusions how to make institutions more accepted and thereby (hopefully) more effective. As an example, if I consider the tax rules and the fiscal authorities to be legitimate, I might be more likely to pay my tax liabilities.

Studying institutions from a cultural perspective may also help not just to understand the institutions, but to understand the culture itself. The University of Copenhagen describes the connection quite well in the syllabus of one of its courses:

“They are also, eo ipso, inherent carriers and producers of cultural histories, “local knowledge”, civilisational assumptions, and identitarian trajectories at both national and regional levels – manifestations, in other words, of often very different political cultures. They embody specific visions of what constitutes the good life, the right kind of morality, and the most appropriate forms of education and socialization in any given social order. Whether we think of governments, judicial or electoral systems, public bureaucracies, social welfare institutions, environmental policies, or educational organizations, they all act as “path-dependent” caretakers of socially desirable identities, rights and duties, worldviews, and interactions between citizens and state.”