The infinite is not necessarily an abyss. It is not necessarily frightening. We have a “taste for the infinite” (“Poème du Hashish," 402). It can follow a wrong route and find a derivative in hashish, or other artificial paradises, instead of leading us to the contemplation of the correspondences of Nature. But art, and poetry, come from this taste of the infinite. They represent its true accomplishment. It is the function of art to make visible the infinite inside the finite. As Baudelaire writes, in a passage concerning the painter Delacroix, 

C’est l’infini dans le fini. C’est le rêve ! Et je n’entends pas par ce mot les capharnaüms de la nuit, mais la vision produite par une intense méditation, ou, dans les cerveaux moins fertiles, par un excitant artificiel. ("Salon 1859," II, 636-7)

In his book, Baudelaire devant l'innombrable, A. Compagnon argues that the infinite only becomes an abyss when it lacks the frame, the border, which the painting, or the poem, necessarily gives to it. Thus, Compagnon sets, on one hand, the borderless sea of the poem “The Seven Old Men,” which leads to madness:  

And, on the other hand, the infinitely and eternally pleasant sea of Baudelaire's diary:

Pourquoi le spectacle de la mer est-il si infiniment et si éternellement agréable ? Parce que la mer offre à la fois l'idée de l'immensité et du mouvement. Six ou sept lieuex représentent pour l'homme le rayon de l'infini. Voilà un infini diminutif. Qu'importe s'il suffit à suggérer l'idée de l'infini total ? Douze ou quatorze lieues (sur le diamètre) douze ou quatorze lieues de liquide en mouvement suffisent pour donner la plus haute idée de beauté qui soit offerte à l'homme sur son habitacle transitoire. ("Mon cœur mis à nu," 696)

The sea is infinite in a circle of twelve or fourteen leagues, as the Nature of the poem "Correspondences" is infinite in the space of the twelve feet of the Alexandrine, the usual form of Baudelaire's verse. In such a frame, the infinite no longer opens itself into an abyss, producing fright or madness. It is beauty itself.

The problem comes with hashish. Baudelaire insists on the expansion of time produced by hashish. The user is literally carried away by the train of his thoughts. It is not that time flies. Quite on the contrary, these thoughts take no time, or almost no time. A minute on the clock represents a whole lifetime of thoughts:

Par bonheur, cette interminable imagination n'a duré qu'une minute, car un intervalle de lucidité avec un grand effort vous a permis d'examiner la pendule. Mais un autre courant d'idées vous emporte ; il vous roulera une minute encore dans son tourbillon vivant, et cette autre minute sera une autre éternité. Car les proportions du temps et de l'être sont dérangées par la multitude innombrable et par l'intensité des sensations et des idées. On vit plusieurs vies d'homme en l'espace d'une heure.  ("Poème du Hashish," 420)

But these eternities that open up in the limits of a single minute may produce a terror in the mind of the user. They are, in fact, like the sea of the seven old men, "monstrous" (414, 432). Nevertheless, they are enclosed in between borders, the two sharp clicks of the needle on the clock passing from one mark to another.

Thus, what makes an abyss is not the absence of border but rather the fact that the numbers that measure the interval between the borders go wrong. In this case, they divide the minute into seconds, and these into an infinity of instants. It is one way in which numbers may go wrong. There may be other ways. In a curious passage of the "Poem of Hashish,” which reminds us of the note from Fusées (and the hallucination invading the domain of pure reasoning), Baudelaire evokes the emergence of numbers in the mind of the user. It starts with the analogies, like the correspondences of nature:

Ces analogies revêtent alors une vivacité inaccoutumée ; elles pénètrent, elles envahissent, elles accablent l'esprit de leur caractère despotique. Les notes musicales deviennent des nombres, et si votre esprit est doué de quelque aptitude mathématique, la mélodie, l'harmonie écoutée, tout en gardant son caractère voluptueux et sensuel, se transforme en une vaste opération mathématique, où les nombres engendrent des nombres. ("Poème du Hashish," 419)

Baudelaire does not say that this vast operation in which the mind is submerged cannot deviate from our usual arithmetical paths. What is remarkable is the cogency of the operation, the drive with which it carries away the user. But, as we saw, numbers when left alone may go wrong, as if, for some obscure reason, two and two made three, or 1882 and 28 added up to 1898. The numbers of hashish may turn into yet another abyss. 

In any case, poetry appears to be a method for framing the abyss of existence into an indivisible, irreducible metrics. Because the twelve feet of the alexandrine cannot divide as the minutes of the time in the hashish pipe, and the words from which the numbers are abstracted keep their arithmetics straight.

Thus there is in poetry a remedy for vertigo. It turns the abysses that move along with us into an "infinitely and eternally pleasant sea."