Songs of songbird species such as Bengalese finch consist of sequences of syllables. While syllables are temporally stereotypical, syllable sequences can vary and follow complex, probabilistic transition rules. Recent experiments and computational models suggest that a syllable is encoded in a chain network of projection neurons in premotor nucleus HVC (proper name). Precisely timed spikes propagate along the chain, driving vocalization of the syllable through downstream nuclei. However, the neural basis of the probabilistic transitions between the syllables is not understood. Here we propose that variable syllable sequences are generated through spike propagations in a network in HVC in which the syllable-encoding chain networks are connected into a branching chain pattern. The neurons mutually inhibit each other through the inhibitory HVC interneurons, and are driven by external inputs from nuclei upstream of HVC. At a branching point that connects the final group of a chain to the first groups of several chains, the spike activity selects one branch to continue the propagation. The selection is probabilistic, and is due to the winner-take-all mechanism mediated by the inhibition and noise. The transitions between the chains are Markovian. If the same syllable can be driven by multiple chains, the generated syllable sequences are statistically described by partially observable Markov models. We suggest that the syntax of birdsong syllable sequences is embedded in the connection patterns of HVC projection neurons.